00010000:00010000:0.0:1789760693.535179:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53b1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.535197:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760693.535202:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.535208:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53b1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17383424->17387519] (req 17383424->17387519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.535222:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53b1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17383424->17387519] (req 17383424->17387519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760693.677644:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508218368 00000100:00100000:0.0:1789760693.677653:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508218368 00000100:00100000:0.0:1789760693.677705:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 672640061629 time 672723722356 state 0x0 ref 1 00000100:00100000:0.0:1789760693.677719:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 672723982772 time 672723722356 state 0x1 ref 1 00000100:00100000:0.0:1789760693.677750:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 672723982772 time 672782459870 state 0x0 ref 1 00000100:00100000:0.0:1789760693.677761:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508218368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760693.677817:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17391616, mtime: 0x6aad94b6, atime: 0x0, ctime: 0x6aad94b6, blocks: 0x8807 00010000:00010000:0.0:1789760693.677826:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53b29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.677842:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=672s v=(0 0) 00010000:00010000:0.0:1789760693.677846:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760693.677853:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53b29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17391616->17395711] (req 17391616->17395711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760693.677873:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53b29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17391616->17395711] (req 17391616->17395711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760694.184519:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760694.226234:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760694.342274:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760694.342361:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760694.342380:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760694.342397:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760694.342416:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00080000:0.0:1789760694.345791:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760694.345796:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000001:00080000:0.0:1789760694.760874:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971577 is committed 00000100:00100000:0.0:1789760694.773880:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760694.869117:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971578 is committed 00000100:00100000:0.0:1789760694.874160:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508220672 00000100:00100000:0.0:1789760694.874168:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508220672 00000100:00100000:0.0:1789760694.874223:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 673789141447 time 673878915139 state 0x0 ref 1 00000100:00100000:0.0:1789760694.874237:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 673879175555 time 673878915139 state 0x1 ref 1 00000100:00100000:0.0:1789760694.874315:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 673879175555 time 673979024278 state 0x0 ref 1 00000100:00100000:0.0:1789760694.874328:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508220672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760694.874462:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17444864, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8807 00010000:00010000:0.0:1789760694.874471:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53b84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.874486:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=673s v=(0 0) 00010000:00010000:0.0:1789760694.874490:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760694.874495:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53b84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17444864->17448959] (req 17444864->17448959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760694.874508:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53b84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17444864->17448959] (req 17444864->17448959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760695.037156:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17453056 -> 17457152 00010000:00010000:0.0:1789760695.037171:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53b92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.037183:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53b92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.037192:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53b92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.037199:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760695.037203:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53b92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.037212:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53b92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.037233:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53b92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.037242:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53b92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17453056->17457151] (req 17453056->17457151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.037503:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508220928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45851us (46250us total) trans 4294971581 rc 0/0 00000100:00100000:0.0:1789760695.037734:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4293 00000100:00100000:0.0:1789760695.037744:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674047680306 time 674096347615 state 0x2 ref 0 00010000:00010000:0.0:1789760695.040063:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760695.041617:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508221056 00000100:00100000:0.0:1789760695.041627:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508221056 00000100:00100000:0.0:1789760695.041696:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674047680306 time 674096347615 state 0x0 ref 1 00000100:00100000:0.0:1789760695.041711:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674096608031 time 674096347615 state 0x1 ref 1 00000100:00100000:0.0:1789760695.041792:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674096608031 time 674146501834 state 0x0 ref 1 00000100:00100000:0.0:1789760695.041805:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508221056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760695.041861:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17457152, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8807 00010000:00010000:0.0:1789760695.041870:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53b99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.041885:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760695.041889:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.041896:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53b99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.041910:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53b99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760695.118241:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17457152 -> 17461248 00010000:00010000:0.0:1789760695.118256:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53b99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.118269:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53b99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.118279:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53b99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.118590:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760695.118596:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53b99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.118606:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53b99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.118626:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53b99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.118634:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53b99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17457152->17461247] (req 17457152->17461247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.118736:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508221056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 76928us (77537us total) trans 4294971582 rc 0/0 00000100:00100000:0.0:1789760695.118747:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4294 00000100:00100000:0.0:1789760695.118756:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674096608031 time 674146501834 state 0x2 ref 0 00000100:00100000:0.0:1789760695.180938:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534049280 00000100:00100000:0.0:1789760695.180949:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534049280 00000100:00100000:0.0:1789760695.184618:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534050432 00000100:00100000:0.0:1789760695.184628:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534050432 00000100:00100000:0.0:1789760695.184662:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 803 00000100:00100000:0.0:1789760695.184672:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3475:x1876699534049536:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760695.185184:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3475:x1876699534049536:12345-0@lo:400:kworker.0 Request processed in 513us (16969us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760695.185191:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 803 00000100:00100000:0.0:1789760695.227234:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508221312 00000100:00100000:0.0:1789760695.227244:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508221312 00000100:00100000:0.0:1789760695.227347:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674146762250 time 674232064523 state 0x0 ref 1 00000100:00100000:0.0:1789760695.227362:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674232324939 time 674232064523 state 0x1 ref 1 00000100:00100000:0.0:1789760695.227384:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674232324939 time 674332095061 state 0x0 ref 1 00000100:00100000:0.0:1789760695.227397:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508221312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760695.227453:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17465344, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8807 00010000:00010000:0.0:1789760695.227462:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53ba7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.227478:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760695.227483:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.227489:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53ba7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17465344->17469439] (req 17465344->17469439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.227503:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53ba7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17465344->17469439] (req 17465344->17469439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.275869:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508221440 00000100:00100000:0.0:1789760695.275878:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508221440 00000100:00100000:0.0:1789760695.275925:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674232324939 time 674332095061 state 0x0 ref 1 00000100:00100000:0.0:1789760695.275939:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674332355477 time 674332095061 state 0x1 ref 1 00000100:00100000:0.0:1789760695.275964:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674332355477 time 674380675574 state 0x0 ref 1 00000100:00100000:0.0:1789760695.275977:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508221440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760695.276023:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17469440, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8807 00010000:00010000:0.0:1789760695.276032:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.276047:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760695.276051:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.276057:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17469440->17473535] (req 17469440->17473535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.276076:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17469440->17473535] (req 17469440->17473535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.519006:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508221696 00000100:00100000:0.0:1789760695.519019:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508221696 00000100:00100000:0.0:1789760695.519079:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674380935990 time 674531799078 state 0x0 ref 1 00000100:00100000:0.0:1789760695.519094:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674532059494 time 674531799078 state 0x1 ref 1 00000100:00100000:0.0:1789760695.519124:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674532059494 time 674623833972 state 0x0 ref 1 00000100:00100000:0.0:1789760695.519136:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508221696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760695.519182:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17477632, mtime: 0x6aad94b7, atime: 0x0, ctime: 0x6aad94b7, blocks: 0x8807 00010000:00010000:0.0:1789760695.519190:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bbc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.519206:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760695.519211:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.519216:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.519230:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760695.564176:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17477632 -> 17481728 00002000:00010000:0.0:1789760695.564185:0:14638:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760695 -> 1789760696 00002000:00010000:0.0:1789760695.564188:0:14638:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760695 -> 1789760696 00010000:00010000:0.0:1789760695.564200:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.564213:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bbc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.564222:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.564229:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760695.564234:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.564242:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bbc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.564261:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bbc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.564269:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bbc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17477632->17481727] (req 17477632->17481727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.564391:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508221696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45252us (46998us total) trans 4294971587 rc 0/0 00000100:00100000:0.0:1789760695.564400:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4299 00000100:00100000:0.0:1789760695.564409:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674532059494 time 674623833972 state 0x2 ref 0 00000001:00080000:0.0:1789760695.564461:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971587 is committed 00000100:00100000:0.0:1789760695.568744:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760695.668540:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17481728 -> 17485824 00010000:00010000:0.0:1789760695.668559:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe53bc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.668571:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe53bc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.668581:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe53bc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.668588:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760695.668592:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe53bc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.668601:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe53bc3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.668619:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe53bc3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.668627:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe53bc3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17481728->17485823] (req 17481728->17485823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.668852:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508221824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 99770us (100103us total) trans 4294971588 rc 0/0 00000100:00100000:0.0:1789760695.669017:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4300 00000100:00100000:0.0:1789760695.669028:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674624094388 time 674673769482 state 0x2 ref 0 00000100:00100000:0.0:1789760695.673264:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508221952 00000100:00100000:0.0:1789760695.673274:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508221952 00000100:00100000:0.0:1789760695.673368:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674624094388 time 674673769482 state 0x0 ref 1 00000100:00100000:0.0:1789760695.673383:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674674029898 time 674673769482 state 0x1 ref 1 00000100:00100000:0.0:1789760695.673408:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674674029898 time 674778118007 state 0x0 ref 1 00000100:00100000:0.0:1789760695.673431:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508221952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760695.673479:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17485824, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0x8807 00010000:00010000:0.0:1789760695.673489:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.673515:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760695.673519:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.673526:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.673541:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760695.778219:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17485824 -> 17489920 00010000:00010000:0.0:1789760695.778240:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.778254:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.778263:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.778269:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760695.778274:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.778578:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.779191:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.779201:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53bca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17485824->17489919] (req 17485824->17489919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.780383:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508221952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 106957us (107210us total) trans 4294971589 rc 0/0 00000100:00100000:0.0:1789760695.780396:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4301 00000100:00100000:0.0:1789760695.780408:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674674029898 time 674778118007 state 0x2 ref 0 00000100:00100000:0.0:1789760695.785089:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508222080 00000100:00100000:0.0:1789760695.785099:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508222080 00000100:00100000:0.0:1789760695.785149:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674674029898 time 674778118007 state 0x0 ref 1 00000100:00100000:0.0:1789760695.785164:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674778378423 time 674778118007 state 0x1 ref 1 00000100:00100000:0.0:1789760695.785193:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674778378423 time 674889903106 state 0x0 ref 1 00000100:00100000:0.0:1789760695.785206:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508222080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760695.785250:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17489920, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0x8807 00010000:00010000:0.0:1789760695.785258:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53bd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.785274:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760695.785278:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.785300:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53bd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17489920->17494015] (req 17489920->17494015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.785314:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53bd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17489920->17494015] (req 17489920->17494015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.857561:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508222208 00000100:00100000:0.0:1789760695.857571:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508222208 00000100:00100000:0.0:1789760695.857629:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674778378423 time 674889903106 state 0x0 ref 1 00000100:00100000:0.0:1789760695.857707:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674890163522 time 674889903106 state 0x1 ref 1 00000100:00100000:0.0:1789760695.857840:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674890163522 time 674962549278 state 0x0 ref 1 00000100:00100000:0.0:1789760695.857856:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d48850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508222208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760695.857917:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17494016, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0x8807 00010000:00010000:0.0:1789760695.857926:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53bd8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.857941:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=674s v=(0 0) 00010000:00010000:0.0:1789760695.857946:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760695.857952:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53bd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17494016->17498111] (req 17494016->17498111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760695.857966:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53bd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17494016->17498111] (req 17494016->17498111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760695.965544:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760696.029570:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17498112 -> 17502208 00010000:00010000:0.0:1789760696.029585:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe53bdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.029599:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe53bdf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.029608:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe53bdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.029615:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760696.029621:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe53bdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.029631:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe53bdf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.029717:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe53bdf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.029726:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe53bdf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17498112->17502207] (req 17498112->17502207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.035724:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508222336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 69833us (70173us total) trans 4294971592 rc 0/0 00000100:00100000:0.0:1789760696.035740:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4304 00000100:00100000:0.0:1789760696.035749:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 674962809694 time 675070578936 state 0x2 ref 0 00000100:00100000:0.0:1789760696.044508:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760696.107580:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508222592 00000100:00100000:0.0:1789760696.107590:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508222592 00000100:00100000:0.0:1789760696.107643:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675070839352 time 675149397990 state 0x0 ref 1 00000100:00100000:0.0:1789760696.107659:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675149658406 time 675149397990 state 0x1 ref 1 00000100:00100000:0.0:1789760696.107688:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675149658406 time 675212398824 state 0x0 ref 1 00000100:00100000:0.0:1789760696.107700:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508222592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760696.107751:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17506304, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0x8807 00010000:00010000:0.0:1789760696.107771:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53bed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.107787:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=675s v=(0 0) 00010000:00010000:0.0:1789760696.107791:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.107797:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53bed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17506304->17510399] (req 17506304->17510399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.107809:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53bed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17506304->17510399] (req 17506304->17510399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.200559:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508222720 00000100:00100000:0.0:1789760696.200569:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508222720 00000100:00100000:0.0:1789760696.200620:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675149658406 time 675212398824 state 0x0 ref 1 00000100:00100000:0.0:1789760696.200634:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675212659240 time 675212398824 state 0x1 ref 1 00000100:00100000:0.0:1789760696.201135:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675212659240 time 675305842551 state 0x0 ref 1 00000100:00100000:0.0:1789760696.201152:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508222720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760696.201199:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17510400, mtime: 0x6aad94b8, atime: 0x0, ctime: 0x6aad94b8, blocks: 0x8807 00010000:00010000:0.0:1789760696.201208:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bf4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.201224:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=675s v=(0 0) 00010000:00010000:0.0:1789760696.201229:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.201236:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.201250:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760696.319224:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17510400 -> 17514496 00010000:00010000:0.0:1789760696.319245:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.319259:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bf4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.319269:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bf4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.319276:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760696.319281:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bf4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.319312:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bf4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.319342:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bf4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.319356:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53bf4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17510400->17514495] (req 17510400->17514495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.319462:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508222720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 118308us (119003us total) trans 4294971595 rc 0/0 00000100:00100000:0.0:1789760696.319473:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4307 00000100:00100000:0.0:1789760696.319483:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675212659240 time 675305842551 state 0x2 ref 0 00000100:00100000:0.0:1789760696.324466:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760696.463508:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760696.573051:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760696.644668:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17526784 -> 17530880 00002000:00010000:0.0:1789760696.644677:0:14638:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760696 -> 1789760697 00002000:00010000:0.0:1789760696.644680:0:14638:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760696 -> 1789760697 00010000:00010000:0.0:1789760696.644693:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe53c10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.644706:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe53c10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.644715:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe53c10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.644722:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760696.644727:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe53c10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.644736:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe53c10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.644755:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe53c10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.644773:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe53c10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17526784->17530879] (req 17526784->17530879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.644870:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508223232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 71608us (71822us total) trans 4294971599 rc 0/0 00000100:00100000:0.0:1789760696.644881:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4311 00000100:00100000:0.0:1789760696.644891:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675568671649 time 675677955004 state 0x2 ref 0 00002000:00010000:0.0:1789760696.688066:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17530880 -> 17534976 00010000:00010000:0.0:1789760696.688082:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53c17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.688180:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53c17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.688191:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53c17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.688198:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760696.688203:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53c17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.688213:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53c17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.688234:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53c17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.688241:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53c17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17530880->17534975] (req 17530880->17534975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.690045:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508223360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41588us (41911us total) trans 4294971600 rc 0/0 00000100:00100000:0.0:1789760696.690059:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4312 00000100:00100000:0.0:1789760696.690069:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675678215420 time 675753145848 state 0x2 ref 0 00000001:00080000:0.0:1789760696.755738:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971601 is committed 00000100:00100000:0.0:1789760696.765210:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760696.898062:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971602 is committed 00000100:00100000:0.0:1789760696.908864:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508223744 00000100:00100000:0.0:1789760696.908874:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508223744 00000100:00100000:0.0:1789760696.908935:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675805978908 time 675871070476 state 0x0 ref 1 00000100:00100000:0.0:1789760696.908951:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675871330892 time 675871070476 state 0x1 ref 1 00000100:00100000:0.0:1789760696.908979:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675871330892 time 676013689171 state 0x0 ref 1 00000100:00100000:0.0:1789760696.908994:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886362300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508223744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760696.909039:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17543168, mtime: 0x6aad94b9, atime: 0x0, ctime: 0x6aad94b9, blocks: 0x8807 00010000:00010000:0.0:1789760696.909048:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53c2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.909063:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760696.909067:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760696.909073:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53c2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.909087:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53c2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760696.982167:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17543168 -> 17547264 00010000:00010000:0.0:1789760696.982202:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53c2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.982215:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53c2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.982223:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53c2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.982229:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760696.982233:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53c2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.982242:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53c2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.982260:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53c2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760696.982268:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53c2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17543168->17547263] (req 17543168->17547263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760696.982403:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886362300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508223744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 73406us (73675us total) trans 4294971603 rc 0/0 00000100:00100000:0.0:1789760696.982423:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4315 00000100:00100000:0.0:1789760696.982433:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 675871330892 time 676013689171 state 0x2 ref 0 00000001:00080000:0.0:1789760696.982473:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971603 is committed 00000100:00100000:0.0:1789760696.986733:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760697.054622:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971604 is committed 00000100:00100000:0.0:1789760697.129918:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760697.285127:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760697.365459:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971610 is committed 00000001:00080000:0.0:1789760697.436109:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971611 is committed 00000100:00100000:0.0:1789760697.443390:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760697.661551:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760697.746586:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760697.884841:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971617 is committed 00000100:00100000:0.0:1789760697.887890:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508225664 00000100:00100000:0.0:1789760697.887899:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508225664 00000100:00100000:0.0:1789760697.887960:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 676851766391 time 676932528349 state 0x0 ref 1 00000100:00100000:0.0:1789760697.887974:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 676932788765 time 676932528349 state 0x1 ref 1 00000100:00100000:0.0:1789760697.888005:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 676932788765 time 676992714969 state 0x0 ref 1 00000100:00100000:0.0:1789760697.888017:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508225664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760697.888064:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17604608, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8807 00010000:00010000:0.0:1789760697.888073:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53c95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.888101:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=676s v=(0 0) 00010000:00010000:0.0:1789760697.888106:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760697.888113:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53c95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.888128:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53c95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760697.944263:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971618 is committed 00002000:00010000:0.0:1789760697.944527:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17604608 -> 17608704 00010000:00010000:0.0:1789760697.944542:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53c95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.944554:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53c95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.944568:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53c95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.944574:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760697.944578:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53c95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.944586:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53c95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.944606:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53c95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760697.944613:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53c95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17604608->17608703] (req 17604608->17608703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760697.944838:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508225664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56817us (57146us total) trans 4294971618 rc 0/0 00000100:00100000:0.0:1789760697.944849:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4330 00000100:00100000:0.0:1789760697.944860:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 676932788765 time 676992714969 state 0x2 ref 0 00000100:00100000:0.0:1789760697.958598:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00010000:00010000:0.0:1789760698.114845:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760698.116271:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508226048 00000100:00100000:0.0:1789760698.116281:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508226048 00000100:00100000:0.0:1789760698.116373:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677063931701 time 677157501783 state 0x0 ref 1 00000100:00100000:0.0:1789760698.116387:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677157762199 time 677157501783 state 0x1 ref 1 00000100:00100000:0.0:1789760698.116424:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677157762199 time 677221133662 state 0x0 ref 1 00000100:00100000:0.0:1789760698.116436:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508226048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760698.116481:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17616896, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8807 00010000:00010000:0.0:1789760698.116490:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53caa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.116504:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760698.116508:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.116514:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53caa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.116528:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53caa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760698.153874:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17616896 -> 17620992 00010000:00010000:0.0:1789760698.153890:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53caa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.153903:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53caa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.153912:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53caa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.153925:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760698.153930:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53caa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.153939:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53caa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.153957:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53caa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.153964:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53caa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17616896->17620991] (req 17616896->17620991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.154067:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508226048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37626us (37892us total) trans 4294971621 rc 0/0 00000100:00100000:0.0:1789760698.154077:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4333 00000100:00100000:0.0:1789760698.154087:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677157762199 time 677221133662 state 0x2 ref 0 00002000:00010000:0.0:1789760698.212409:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17620992 -> 17625088 00010000:00010000:0.0:1789760698.212424:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe53cb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.212436:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe53cb1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.212451:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe53cb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.212459:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760698.212464:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe53cb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.212473:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe53cb1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.212495:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe53cb1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.212503:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe53cb1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17620992->17625087] (req 17620992->17625087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.212608:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886363800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508226176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52692us (52967us total) trans 4294971622 rc 0/0 00000100:00100000:0.0:1789760698.212618:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4334 00000100:00100000:0.0:1789760698.212628:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677221394078 time 677264611231 state 0x2 ref 0 00000100:00100000:0.0:1789760698.223094:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508226304 00000100:00100000:0.0:1789760698.223103:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508226304 00000100:00100000:0.0:1789760698.223152:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677221394078 time 677264611231 state 0x0 ref 1 00000100:00100000:0.0:1789760698.223166:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677264871647 time 677264611231 state 0x1 ref 1 00000100:00100000:0.0:1789760698.223203:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677264871647 time 677327913905 state 0x0 ref 1 00000100:00100000:0.0:1789760698.223214:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508226304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760698.223258:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17625088, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8807 00010000:00010000:0.0:1789760698.223267:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53cb8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.223283:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760698.223320:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.223327:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53cb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17625088->17629183] (req 17625088->17629183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.223341:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53cb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17625088->17629183] (req 17625088->17629183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.241558:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499bfed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534050944:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760698.241625:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760698.241690:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534050944 00000100:00100000:0.0:1789760698.241697:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534050944 00000100:00100000:0.0:1789760698.241742:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:0.0:1789760698.241750:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534050944:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760698.241785:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789760698.241827:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfec700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534050944:12345-0@lo:41:osp-pre-1-0.0 Request processed in 77us (201us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760698.241834:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:0.0:1789760698.241879:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499bfed500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534050944:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760698.281004:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508226432 00000100:00100000:0.0:1789760698.281013:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508226432 00000100:00100000:0.0:1789760698.281064:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677264871647 time 677327913905 state 0x0 ref 1 00000100:00100000:0.0:1789760698.281078:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677328174321 time 677327913905 state 0x1 ref 1 00000100:00100000:0.0:1789760698.281108:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677328174321 time 677385818782 state 0x0 ref 1 00000100:00100000:0.0:1789760698.281120:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508226432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760698.281169:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17629184, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8807 00010000:00010000:0.0:1789760698.281178:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53cbf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.281195:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760698.281199:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.281204:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53cbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.281217:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53cbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760698.335166:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17629184 -> 17633280 00010000:00010000:0.0:1789760698.335196:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53cbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.335210:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53cbf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.335219:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53cbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.335226:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760698.335230:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53cbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.335239:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53cbf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.335257:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53cbf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.335264:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53cbf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17629184->17633279] (req 17629184->17633279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.335412:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508226432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54279us (54566us total) trans 4294971624 rc 0/0 00000100:00100000:0.0:1789760698.335423:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4336 00000100:00100000:0.0:1789760698.335433:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677328174321 time 677385818782 state 0x2 ref 0 00000100:00100000:0.0:1789760698.338885:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760698.445730:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508226816 00000100:00100000:0.0:1789760698.445739:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508226816 00000100:00100000:0.0:1789760698.445891:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677444048475 time 677509162749 state 0x0 ref 1 00000100:00100000:0.0:1789760698.445909:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677509423165 time 677509162749 state 0x1 ref 1 00000100:00100000:0.0:1789760698.445941:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677509423165 time 677550651551 state 0x0 ref 1 00000100:00100000:0.0:1789760698.445954:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508226816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760698.445998:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17641472, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8807 00010000:00010000:0.0:1789760698.446006:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53cd4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.446022:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760698.446026:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.446033:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53cd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17641472->17645567] (req 17641472->17645567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.446046:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53cd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17641472->17645567] (req 17641472->17645567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.505664:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508226816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59706us (60033us total) trans 4294971627 rc 0/0 00000100:00100000:0.0:1789760698.505674:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4339 00000100:00100000:0.0:1789760698.505685:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677509423165 time 677550651551 state 0x2 ref 0 00000100:00100000:0.0:1789760698.509246:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760698.519566:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677509423165 time 677550651551 state 0x0 ref 1 00000100:00100000:0.0:1789760698.519581:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677550911967 time 677550651551 state 0x1 ref 1 00000100:00100000:0.0:1789760698.519608:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677550911967 time 677624318766 state 0x0 ref 1 00000100:00100000:0.0:1789760698.519622:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508226944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760698.519671:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17645568, mtime: 0x6aad94ba, atime: 0x0, ctime: 0x6aad94ba, blocks: 0x8807 00010000:00010000:0.0:1789760698.519681:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53cdb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.519696:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760698.519701:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.519707:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53cdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17645568->17649663] (req 17645568->17649663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.519722:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53cdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17645568->17649663] (req 17645568->17649663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.641897:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508227200 00000100:00100000:0.0:1789760698.641908:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508227200 00000100:00100000:0.0:1789760698.641965:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677624579182 time 677682239620 state 0x0 ref 1 00000100:00100000:0.0:1789760698.641981:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677682500036 time 677682239620 state 0x1 ref 1 00000100:00100000:0.0:1789760698.642011:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677682500036 time 677746721242 state 0x0 ref 1 00000100:00100000:0.0:1789760698.642023:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508227200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760698.642071:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17653760, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8807 00010000:00010000:0.0:1789760698.642080:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53ce9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.642096:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760698.642100:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.642106:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53ce9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17653760->17657855] (req 17653760->17657855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.642119:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53ce9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17653760->17657855] (req 17653760->17657855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760698.709263:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971630 is committed 00002000:00010000:0.0:1789760698.763103:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17657856 -> 17661952 00010000:00010000:0.0:1789760698.763123:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53cf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.763136:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53cf0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.763146:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53cf0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.763153:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760698.763158:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53cf0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.763168:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53cf0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.763191:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53cf0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.763199:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53cf0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17657856->17661951] (req 17657856->17661951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.763503:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508227328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49726us (50021us total) trans 4294971631 rc 0/0 00000100:00100000:0.0:1789760698.763514:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4343 00000100:00100000:0.0:1789760698.763525:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677746981658 time 677818470380 state 0x2 ref 0 00000001:00080000:0.0:1789760698.867998:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971633 is committed 00000100:00100000:0.0:1789760698.873675:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508227712 00000100:00100000:0.0:1789760698.873688:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508227712 00000100:00100000:0.0:1789760698.873857:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677872840674 time 677924386622 state 0x0 ref 1 00000100:00100000:0.0:1789760698.873871:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677924647038 time 677924386622 state 0x1 ref 1 00000100:00100000:0.0:1789760698.874003:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677924647038 time 677978712230 state 0x0 ref 1 00000100:00100000:0.0:1789760698.874017:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508227712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760698.874064:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17670144, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8807 00010000:00010000:0.0:1789760698.874137:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.874160:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=677s v=(0 0) 00010000:00010000:0.0:1789760698.874165:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760698.874171:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.874185:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760698.924686:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17670144 -> 17674240 00010000:00010000:0.0:1789760698.924818:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.924832:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.924841:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.924849:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760698.924853:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.924861:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.924883:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760698.924892:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17670144->17674239] (req 17670144->17674239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760698.925028:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508227712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51009us (51780us total) trans 4294971634 rc 0/0 00000100:00100000:0.0:1789760698.925039:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4346 00000100:00100000:0.0:1789760698.925048:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 677924647038 time 677978712230 state 0x2 ref 0 00000001:00080000:0.0:1789760698.989710:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971635 is committed 00000100:00100000:0.0:1789760698.994176:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760699.086809:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508228096 00000100:00100000:0.0:1789760699.086818:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508228096 00000100:00100000:0.0:1789760699.086984:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678035153189 time 678099345902 state 0x0 ref 1 00000100:00100000:0.0:1789760699.086999:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678099606318 time 678099345902 state 0x1 ref 1 00000100:00100000:0.0:1789760699.087059:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678099606318 time 678191768398 state 0x0 ref 1 00000100:00100000:0.0:1789760699.087080:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508228096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760699.087127:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17682432, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8807 00010000:00010000:0.0:1789760699.087137:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.087153:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760699.087157:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.087164:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.087177:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760699.176419:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17682432 -> 17686528 00010000:00010000:0.0:1789760699.176434:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.176446:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.176454:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.176461:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760699.176466:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.176476:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.176493:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.176501:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17682432->17686527] (req 17682432->17686527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.176599:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508228096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 89521us (89928us total) trans 4294971637 rc 0/0 00000100:00100000:0.0:1789760699.176608:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4349 00000100:00100000:0.0:1789760699.176617:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678099606318 time 678191768398 state 0x2 ref 0 00000100:00100000:0.0:1789760699.252190:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508228352 00000100:00100000:0.0:1789760699.252200:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508228352 00000100:00100000:0.0:1789760699.252255:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678192028814 time 678286049682 state 0x0 ref 1 00000100:00100000:0.0:1789760699.252271:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678286310098 time 678286049682 state 0x1 ref 1 00000100:00100000:0.0:1789760699.252321:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678286310098 time 678357031401 state 0x0 ref 1 00000100:00100000:0.0:1789760699.252334:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508228352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760699.252381:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17690624, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8807 00010000:00010000:0.0:1789760699.252400:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.252415:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760699.252419:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.252425:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17690624->17694719] (req 17690624->17694719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.252437:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe53d28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17690624->17694719] (req 17690624->17694719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760699.337521:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971639 is committed 00000100:00100000:0.0:1789760699.343024:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508228480 00000100:00100000:0.0:1789760699.343035:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508228480 00000100:00100000:0.0:1789760699.343165:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678286310098 time 678357031401 state 0x0 ref 1 00000100:00100000:0.0:1789760699.343180:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678357291817 time 678357031401 state 0x1 ref 1 00000100:00100000:0.0:1789760699.343213:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678357291817 time 678447924160 state 0x0 ref 1 00000100:00100000:0.0:1789760699.343226:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508228480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760699.343419:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17694720, mtime: 0x6aad94bb, atime: 0x0, ctime: 0x6aad94bb, blocks: 0x8807 00010000:00010000:0.0:1789760699.343429:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53d2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.343445:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760699.343449:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.343456:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53d2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.343469:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53d2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760699.387625:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17694720 -> 17698816 00010000:00010000:0.0:1789760699.387642:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53d2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.387660:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53d2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.387669:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53d2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.387677:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760699.387682:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53d2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.387691:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53d2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.387711:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53d2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.387719:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53d2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17694720->17698815] (req 17694720->17698815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.387953:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508228480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44724us (45117us total) trans 4294971640 rc 0/0 00000100:00100000:0.0:1789760699.387963:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4352 00000100:00100000:0.0:1789760699.387972:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678357291817 time 678447924160 state 0x2 ref 0 00000100:00100000:0.0:1789760699.393610:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760699.424612:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971641 is committed 00000100:00100000:0.0:1789760699.428956:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760699.467054:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17702912 -> 17707008 00010000:00010000:0.0:1789760699.467068:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe53d3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.467083:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe53d3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.467092:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe53d3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.467100:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760699.467105:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe53d3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.467113:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe53d3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.467132:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe53d3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.467141:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe53d3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17702912->17707007] (req 17702912->17707007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.467260:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508228736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38041us (38311us total) trans 4294971642 rc 0/0 00000100:00100000:0.0:1789760699.467281:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4354 00000100:00100000:0.0:1789760699.467448:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678498925045 time 678533913930 state 0x2 ref 0 00002000:00010000:0.0:1789760699.498959:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17707008 -> 17711104 00002000:00010000:0.0:1789760699.498964:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760699 -> 1789760700 00002000:00010000:0.0:1789760699.498966:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760699 -> 1789760700 00010000:00010000:0.0:1789760699.498980:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe53d44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.498992:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe53d44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.499002:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe53d44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.499009:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760699.499013:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe53d44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.499023:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe53d44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.499042:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe53d44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.499050:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe53d44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17707008->17711103] (req 17707008->17711103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.499147:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886361c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508229504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26878us (27086us total) trans 4294971643 rc 0/0 00000100:00100000:0.0:1789760699.499157:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4355 00000100:00100000:0.0:1789760699.499167:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678534174346 time 678576964189 state 0x2 ref 0 00000100:00100000:0.0:1789760699.546253:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508229760 00000100:00100000:0.0:1789760699.546270:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508229760 00000100:00100000:0.0:1789760699.546475:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678577224605 time 678608440891 state 0x0 ref 1 00000100:00100000:0.0:1789760699.546489:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678608701307 time 678608440891 state 0x1 ref 1 00000100:00100000:0.0:1789760699.546516:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678608701307 time 678651227186 state 0x0 ref 1 00000100:00100000:0.0:1789760699.546529:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508229760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760699.546574:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17715200, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8807 00010000:00010000:0.0:1789760699.546583:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53d52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.546599:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760699.546603:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.546609:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53d52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.546622:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53d52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760699.578039:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17715200 -> 17719296 00010000:00010000:0.0:1789760699.578056:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53d52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.578068:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53d52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.578077:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53d52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.578084:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760699.578088:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53d52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.578097:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53d52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.578116:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53d52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.578123:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53d52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17715200->17719295] (req 17715200->17719295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.578214:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508229760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31682us (32110us total) trans 4294971645 rc 0/0 00000100:00100000:0.0:1789760699.578228:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4357 00000100:00100000:0.0:1789760699.578237:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678608701307 time 678651227186 state 0x2 ref 0 00000100:00100000:0.0:1789760699.582022:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760699.653503:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508230144 00000100:00100000:0.0:1789760699.653513:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508230144 00000100:00100000:0.0:1789760699.653564:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678687177913 time 678722861320 state 0x0 ref 1 00000100:00100000:0.0:1789760699.653579:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678723121736 time 678722861320 state 0x1 ref 1 00000100:00100000:0.0:1789760699.653654:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678723121736 time 678758363391 state 0x0 ref 1 00000100:00100000:0.0:1789760699.653667:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d44050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508230144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760699.653714:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17727488, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8807 00010000:00010000:0.0:1789760699.653723:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53d67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.653799:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760699.653804:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.653810:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53d67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17727488->17731583] (req 17727488->17731583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.653826:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53d67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17727488->17731583] (req 17727488->17731583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.695986:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760699.766888:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508230528 00000100:00100000:0.0:1789760699.766897:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508230528 00000100:00100000:0.0:1789760699.766945:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678801399762 time 678837838613 state 0x0 ref 1 00000100:00100000:0.0:1789760699.766960:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678838099029 time 678837838613 state 0x1 ref 1 00000100:00100000:0.0:1789760699.766989:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678838099029 time 678871699282 state 0x0 ref 1 00000100:00100000:0.0:1789760699.767000:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508230528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760699.767046:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17739776, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8807 00010000:00010000:0.0:1789760699.767055:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53d7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.767070:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760699.767074:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.767080:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53d7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17739776->17743871] (req 17739776->17743871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.767095:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53d7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17739776->17743871] (req 17739776->17743871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.798327:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508230656 00000100:00100000:0.0:1789760699.798337:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508230656 00000100:00100000:0.0:1789760699.798401:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678838099029 time 678871699282 state 0x0 ref 1 00000100:00100000:0.0:1789760699.798416:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678871959698 time 678871699282 state 0x1 ref 1 00000100:00100000:0.0:1789760699.798444:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678871959698 time 678903155072 state 0x0 ref 1 00000100:00100000:0.0:1789760699.798457:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508230656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760699.798512:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17743872, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8807 00010000:00010000:0.0:1789760699.798520:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.798536:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=678s v=(0 0) 00010000:00010000:0.0:1789760699.798541:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.798548:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.798561:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760699.831180:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17743872 -> 17747968 00010000:00010000:0.0:1789760699.831202:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.831215:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.831225:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.831232:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760699.831237:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.831247:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.831277:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.831314:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17743872->17747967] (req 17743872->17747967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.831419:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508230656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32958us (33198us total) trans 4294971652 rc 0/0 00000100:00100000:0.0:1789760699.831430:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4364 00000100:00100000:0.0:1789760699.831440:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678871959698 time 678903155072 state 0x2 ref 0 00002000:00010000:0.0:1789760699.872249:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17747968 -> 17752064 00010000:00010000:0.0:1789760699.872275:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53d8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.872314:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53d8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.872324:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53d8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.872340:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760699.872345:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53d8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.872354:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53d8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.872374:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53d8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.872396:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe53d8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17747968->17752063] (req 17747968->17752063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760699.872487:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508230784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36816us (37019us total) trans 4294971653 rc 0/0 00000100:00100000:0.0:1789760699.872497:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4365 00000100:00100000:0.0:1789760699.872506:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678903415488 time 678940365770 state 0x2 ref 0 00000100:00100000:0.0:1789760699.908104:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508231040 00000100:00100000:0.0:1789760699.908114:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508231040 00000100:00100000:0.0:1789760699.908167:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678940626186 time 678982425938 state 0x0 ref 1 00000100:00100000:0.0:1789760699.908182:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678982686354 time 678982425938 state 0x1 ref 1 00000100:00100000:0.0:1789760699.908255:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 678982686354 time 679012965318 state 0x0 ref 1 00000100:00100000:0.0:1789760699.908277:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508231040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760699.908514:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17756160, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8807 00010000:00010000:0.0:1789760699.908524:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.908539:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760699.908543:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760699.908549:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17756160->17760255] (req 17756160->17760255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760699.908564:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53d98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17756160->17760255] (req 17756160->17760255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.033708:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971657 is committed 00002000:00010000:0.0:1789760700.088182:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17768448 -> 17772544 00010000:00010000:0.0:1789760700.088198:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe53dad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.088210:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe53dad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.088219:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe53dad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.088227:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760700.088231:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe53dad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.088240:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe53dad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.088419:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe53dad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.088428:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe53dad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17768448->17772543] (req 17768448->17772543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.088561:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508231424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49365us (49825us total) trans 4294971658 rc 0/0 00000100:00100000:0.0:1789760700.088571:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4370 00000100:00100000:0.0:1789760700.088581:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679096341374 time 679143884294 state 0x2 ref 0 00000100:00100000:0.0:1789760700.092636:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760700.171534:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17772544 -> 17776640 00010000:00010000:0.0:1789760700.171550:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53db4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.171564:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53db4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.171573:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53db4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.171581:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760700.171586:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53db4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.171600:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53db4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.171622:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53db4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.171630:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53db4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17772544->17776639] (req 17772544->17776639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.171835:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508231552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 78777us (79205us total) trans 4294971659 rc 0/0 00000100:00100000:0.0:1789760700.171845:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4371 00000100:00100000:0.0:1789760700.171854:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679144144710 time 679197751680 state 0x2 ref 0 00000100:00100000:0.0:1789760700.258721:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508231808 00000100:00100000:0.0:1789760700.258731:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508231808 00000100:00100000:0.0:1789760700.258793:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679198012096 time 679280576932 state 0x0 ref 1 00000100:00100000:0.0:1789760700.258809:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679280837348 time 679280576932 state 0x1 ref 1 00000100:00100000:0.0:1789760700.258838:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679280837348 time 679363548914 state 0x0 ref 1 00000100:00100000:0.0:1789760700.258850:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508231808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760700.258897:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17780736, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8807 00010000:00010000:0.0:1789760700.258906:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dc2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.258923:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760700.258928:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.258934:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.258949:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.290064:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b6c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534051328:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760700.290094:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760700.294140:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534052096 00000100:00100000:0.0:1789760700.294146:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534052096 00000100:00100000:0.0:1789760700.295519:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534051840 00000100:00100000:0.0:1789760700.295525:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534051840 00002000:00010000:0.0:1789760700.329241:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17780736 -> 17784832 00010000:00010000:0.0:1789760700.329256:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.329268:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dc2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.329278:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.329318:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760700.329323:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.329334:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dc2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.329353:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dc2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.329360:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dc2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17780736->17784831] (req 17780736->17784831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.329463:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508231808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 70609us (70835us total) trans 4294971661 rc 0/0 00000100:00100000:0.0:1789760700.329473:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4373 00000100:00100000:0.0:1789760700.329482:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679280837348 time 679363548914 state 0x2 ref 0 00000100:00100000:0.0:1789760700.332890:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508231936 00000100:00100000:0.0:1789760700.332899:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508231936 00000100:00100000:0.0:1789760700.332949:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679280837348 time 679363548914 state 0x0 ref 1 00000100:00100000:0.0:1789760700.332963:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679363809330 time 679363548914 state 0x1 ref 1 00000100:00100000:0.0:1789760700.332994:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679363809330 time 679437704215 state 0x0 ref 1 00000100:00100000:0.0:1789760700.333006:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508231936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760700.333052:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17784832, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8807 00010000:00010000:0.0:1789760700.333061:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53dc9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.333078:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760700.333082:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.333088:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53dc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.333101:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53dc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17784832->17788927] (req 17784832->17788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.395134:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508232192 00000100:00100000:0.0:1789760700.395145:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508232192 00000100:00100000:0.0:1789760700.395196:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679437964631 time 679469738101 state 0x0 ref 1 00000100:00100000:0.0:1789760700.395210:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679469998517 time 679469738101 state 0x1 ref 1 00000100:00100000:0.0:1789760700.395240:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679469998517 time 679499950268 state 0x0 ref 1 00000100:00100000:0.0:1789760700.395252:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508232192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760700.395337:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17793024, mtime: 0x6aad94bc, atime: 0x0, ctime: 0x6aad94bc, blocks: 0x8807 00010000:00010000:0.0:1789760700.395347:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dd7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.395364:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760700.395369:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.395375:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.395405:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760700.422106:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17793024 -> 17797120 00010000:00010000:0.0:1789760700.422122:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.422134:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dd7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.422143:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.422150:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760700.422154:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.422162:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dd7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.422182:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dd7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.422191:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53dd7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17793024->17797119] (req 17793024->17797119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.422407:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508232192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27032us (27233us total) trans 4294971664 rc 0/0 00000100:00100000:0.0:1789760700.422419:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4376 00000100:00100000:0.0:1789760700.422429:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679469998517 time 679499950268 state 0x2 ref 0 00000001:00080000:0.0:1789760700.515052:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971666 is committed 00000100:00100000:0.0:1789760700.520018:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508232576 00000100:00100000:0.0:1789760700.520029:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508232576 00000100:00100000:0.0:1789760700.520086:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679531797155 time 679576988550 state 0x0 ref 1 00000100:00100000:0.0:1789760700.520103:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679577248966 time 679576988550 state 0x1 ref 1 00000100:00100000:0.0:1789760700.520130:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679577248966 time 679624840695 state 0x0 ref 1 00000100:00100000:0.0:1789760700.520143:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508232576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760700.520188:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17805312, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8807 00010000:00010000:0.0:1789760700.520198:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53dec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.520215:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760700.520219:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.520225:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53dec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.520240:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53dec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760700.605565:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971667 is committed 00002000:00010000:0.0:1789760700.605664:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17805312 -> 17809408 00010000:00010000:0.0:1789760700.605684:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53dec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.605699:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53dec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.605709:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53dec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.605716:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760700.605723:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53dec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.605742:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53dec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.605767:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53dec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.605776:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53dec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17805312->17809407] (req 17805312->17809407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.605895:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508232576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 85747us (86833us total) trans 4294971667 rc 0/0 00000100:00100000:0.0:1789760700.605908:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4379 00000100:00100000:0.0:1789760700.605921:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679577248966 time 679624840695 state 0x2 ref 0 00000100:00100000:0.0:1789760700.609637:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760700.666904:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971668 is committed 00000001:00080000:0.0:1789760700.734095:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971669 is committed 00000100:00100000:0.0:1789760700.738533:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508232960 00000100:00100000:0.0:1789760700.738547:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508232960 00000100:00100000:0.0:1789760700.738620:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679714970768 time 679776595383 state 0x0 ref 1 00000100:00100000:0.0:1789760700.738639:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679776855799 time 679776595383 state 0x1 ref 1 00000100:00100000:0.0:1789760700.738712:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679776855799 time 679843420490 state 0x0 ref 1 00000100:00100000:0.0:1789760700.738791:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508232960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760700.738865:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17817600, mtime: 0x6aad94bd, atime: 0x0, ctime: 0x6aad94bd, blocks: 0x8807 00010000:00010000:0.0:1789760700.738877:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53e01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.738899:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789760700.738905:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760700.738913:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53e01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.738930:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53e01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760700.784197:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17817600 -> 17821696 00010000:00010000:0.0:1789760700.784218:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53e01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.784234:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53e01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.784244:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53e01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.784253:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760700.784259:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53e01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.784270:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53e01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.784487:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53e01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.784498:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53e01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17817600->17821695] (req 17817600->17821695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.784650:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508232960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45918us (46401us total) trans 4294971670 rc 0/0 00000100:00100000:0.0:1789760700.784667:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4382 00000100:00100000:0.0:1789760700.784678:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679776855799 time 679843420490 state 0x2 ref 0 00000100:00100000:0.0:1789760700.933987:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760700.998083:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17829888 -> 17833984 00010000:00010000:0.0:1789760700.998107:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53e16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.998123:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53e16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.998134:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53e16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.998143:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760700.998150:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53e16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.998161:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53e16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.998188:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53e16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760700.998197:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe53e16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17829888->17833983] (req 17829888->17833983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760700.998584:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508233344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59656us (64604us total) trans 4294971673 rc 0/0 00000100:00100000:0.0:1789760700.998597:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4385 00000100:00100000:0.0:1789760700.998608:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 679940111902 time 680043602459 state 0x2 ref 0 00000001:00080000:0.0:1789760701.215281:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971675 is committed 00000100:00100000:0.0:1789760701.233520:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760701.266933:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971676 is committed 00000100:00100000:0.0:1789760701.317449:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760701.350434:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971678 is committed 00000100:00100000:0.0:1789760701.353229:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760701.401541:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971680 is committed 00000100:00100000:0.0:1789760701.412543:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760701.435056:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17862656 -> 17866752 00010000:00010000:0.0:1789760701.435075:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe53e4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.435089:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe53e4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.435154:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe53e4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.435164:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760701.435169:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe53e4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.435185:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe53e4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.435207:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe53e4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.435215:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe53e4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17862656->17866751] (req 17862656->17866751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.435409:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508234368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22616us (22870us total) trans 4294971681 rc 0/0 00000100:00100000:0.0:1789760701.435420:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4393 00000100:00100000:0.0:1789760701.435430:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 680485613272 time 680517486810 state 0x2 ref 0 00000001:00080000:0.0:1789760701.435475:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971681 is committed 00000100:00100000:0.0:1789760701.438573:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760701.467095:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971682 is committed 00002000:00010000:0.0:1789760701.498024:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17870848 -> 17874944 00002000:00010000:0.0:1789760701.498027:0:14638:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760701 -> 1789760702 00002000:00010000:0.0:1789760701.498030:0:14638:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760701 -> 1789760702 00010000:00010000:0.0:1789760701.498041:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe53e5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.498050:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe53e5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.498060:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe53e5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.498065:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760701.498068:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe53e5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.498074:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe53e5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.498089:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe53e5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.498094:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe53e5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17870848->17874943] (req 17870848->17874943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.498177:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508234624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26382us (26734us total) trans 4294971683 rc 0/0 00000100:00100000:0.0:1789760701.498188:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4395 00000100:00100000:0.0:1789760701.498196:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 680543718941 time 680576489878 state 0x2 ref 0 00000100:00100000:0.0:1789760701.502003:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760701.526213:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971684 is committed 00000100:00100000:0.0:1789760701.529859:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508234880 00000100:00100000:0.0:1789760701.529868:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508234880 00000100:00100000:0.0:1789760701.529920:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 680576750294 time 680607425779 state 0x0 ref 1 00000100:00100000:0.0:1789760701.529939:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 680607686195 time 680607425779 state 0x1 ref 1 00000100:00100000:0.0:1789760701.530005:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 680607686195 time 680634714491 state 0x0 ref 1 00000100:00100000:0.0:1789760701.530017:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508234880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760701.530064:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17879040, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9007 00010000:00010000:0.0:1789760701.530073:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53e6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.530090:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760701.530094:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.530100:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53e6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.530113:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53e6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17879040->17883135] (req 17879040->17883135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.555480:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508235008 00000100:00100000:0.0:1789760701.555489:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508235008 00000100:00100000:0.0:1789760701.555539:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 680607686195 time 680634714491 state 0x0 ref 1 00000100:00100000:0.0:1789760701.555553:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 680634974907 time 680634714491 state 0x1 ref 1 00000100:00100000:0.0:1789760701.555579:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 680634974907 time 680660290817 state 0x0 ref 1 00000100:00100000:0.0:1789760701.555588:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508235008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760701.555630:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17883136, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9007 00010000:00010000:0.0:1789760701.555637:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53e71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.555650:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=680s v=(0 0) 00010000:00010000:0.0:1789760701.555654:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.555659:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53e71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17883136->17887231] (req 17883136->17887231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.555670:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe53e71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17883136->17887231] (req 17883136->17887231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.601790:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760701.642208:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17887232 -> 17891328 00010000:00010000:0.0:1789760701.642226:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe53e78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.642239:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe53e78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.642248:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe53e78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.642255:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760701.642260:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe53e78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.642274:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe53e78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.642389:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe53e78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.642398:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe53e78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17887232->17891327] (req 17887232->17891327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.642494:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508235136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40251us (40709us total) trans 4294971687 rc 0/0 00000100:00100000:0.0:1789760701.642505:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4399 00000100:00100000:0.0:1789760701.642515:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 680660551233 time 680706935566 state 0x2 ref 0 00000100:00100000:0.0:1789760701.653548:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760701.733012:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760701.874106:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971692 is committed 00000100:00100000:0.0:1789760701.874533:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508235776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32584us (32789us total) trans 4294971692 rc 0/0 00000100:00100000:0.0:1789760701.874544:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4404 00000100:00100000:0.0:1789760701.874555:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 680880381437 time 680946644953 state 0x2 ref 0 00000100:00100000:0.0:1789760701.877889:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760701.916526:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971693 is committed 00000100:00100000:0.0:1789760701.921499:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760701.972655:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17915904 -> 17920000 00010000:00010000:0.0:1789760701.972675:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53ea9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.972688:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53ea9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.972697:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53ea9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.972704:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760701.972709:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53ea9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.973053:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53ea9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.973077:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53ea9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.973085:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53ea9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17915904->17919999] (req 17915904->17919999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760701.973238:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508236032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51515us (51743us total) trans 4294971694 rc 0/0 00000100:00100000:0.0:1789760701.973251:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4406 00000100:00100000:0.0:1789760701.973262:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 680983609452 time 681026416919 state 0x2 ref 0 00000001:00080000:0.0:1789760701.973430:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971694 is committed 00000100:00100000:0.0:1789760701.987128:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508236160 00000100:00100000:0.0:1789760701.987142:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508236160 00000100:00100000:0.0:1789760701.987218:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 680983609452 time 681026416919 state 0x0 ref 1 00000100:00100000:0.0:1789760701.987238:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681026677335 time 681026416919 state 0x1 ref 1 00000100:00100000:0.0:1789760701.987313:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681026677335 time 681091993181 state 0x0 ref 1 00000100:00100000:0.0:1789760701.987328:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508236160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760701.987396:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17920000, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9007 00010000:00010000:0.0:1789760701.987405:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53eb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.987422:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760701.987427:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760701.987433:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53eb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760701.987448:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53eb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760702.044958:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17920000 -> 17924096 00010000:00010000:0.0:1789760702.044977:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53eb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.044996:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53eb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.045005:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53eb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.045012:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760702.045016:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53eb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.045025:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53eb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.045044:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53eb0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.045051:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53eb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17920000->17924095] (req 17920000->17924095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.045146:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508236160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57817us (59226us total) trans 4294971695 rc 0/0 00000100:00100000:0.0:1789760702.045156:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4407 00000100:00100000:0.0:1789760702.045165:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681026677335 time 681091993181 state 0x2 ref 0 00000001:00080000:0.0:1789760702.089775:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971696 is committed 00000100:00100000:0.0:1789760702.091919:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508236416 00000100:00100000:0.0:1789760702.091930:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508236416 00000100:00100000:0.0:1789760702.091982:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681092253597 time 681155203520 state 0x0 ref 1 00000100:00100000:0.0:1789760702.091996:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681155463936 time 681155203520 state 0x1 ref 1 00000100:00100000:0.0:1789760702.092025:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681155463936 time 681196736102 state 0x0 ref 1 00000100:00100000:0.0:1789760702.092037:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508236416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760702.092082:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17928192, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9007 00010000:00010000:0.0:1789760702.092091:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53ebe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.092106:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760702.092110:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.092120:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53ebe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17928192->17932287] (req 17928192->17932287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.092134:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53ebe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17928192->17932287] (req 17928192->17932287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.129234:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971697 is committed 00000100:00100000:0.0:1789760702.140592:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508236544 00000100:00100000:0.0:1789760702.140602:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508236544 00000100:00100000:0.0:1789760702.231774:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760702.292311:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971700 is committed 00000100:00100000:0.0:1789760702.389046:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681336948258 time 681409839269 state 0x0 ref 1 00000100:00100000:0.0:1789760702.389065:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681410099685 time 681409839269 state 0x1 ref 1 00000100:00100000:0.0:1789760702.389130:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681410099685 time 681493840210 state 0x0 ref 1 00000100:00100000:0.0:1789760702.389144:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508237056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760702.389195:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17948672, mtime: 0x6aad94be, atime: 0x0, ctime: 0x6aad94be, blocks: 0x9007 00010000:00010000:0.0:1789760702.389205:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53ee1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.389222:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760702.389226:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.389231:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53ee1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17948672->17952767] (req 17948672->17952767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.389245:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53ee1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17948672->17952767] (req 17948672->17952767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.445771:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760702.519959:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17952768 -> 17956864 00010000:00010000:0.0:1789760702.519981:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe53ee8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.519995:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe53ee8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.520003:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe53ee8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.520011:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760702.520016:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe53ee8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.520027:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe53ee8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.520048:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe53ee8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.520056:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe53ee8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17952768->17956863] (req 17952768->17956863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.520158:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508237184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 74065us (74390us total) trans 4294971703 rc 0/0 00000100:00100000:0.0:1789760702.520169:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4415 00000100:00100000:0.0:1789760702.520180:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681494100626 time 681550785412 state 0x2 ref 0 00002000:00010000:0.0:1789760702.579884:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17956864 -> 17960960 00002000:00010000:0.0:1789760702.579893:0:14638:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760702 -> 1789760703 00002000:00010000:0.0:1789760702.579896:0:14638:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760702 -> 1789760703 00010000:00010000:0.0:1789760702.579908:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53eef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.579922:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53eef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.579932:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53eef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.579939:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760702.579944:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53eef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.579952:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53eef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.579971:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53eef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.579979:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe53eef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17956864->17960959] (req 17956864->17960959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.580076:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508237312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56424us (56800us total) trans 4294971704 rc 0/0 00000100:00100000:0.0:1789760702.580087:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4416 00000100:00100000:0.0:1789760702.580097:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681551045828 time 681628347577 state 0x2 ref 0 00002000:00010000:0.0:1789760702.640759:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17960960 -> 17965056 00010000:00010000:0.0:1789760702.640776:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53ef6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.640796:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53ef6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.640805:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53ef6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.640812:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760702.640817:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53ef6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.640827:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53ef6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.640845:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53ef6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.640853:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe53ef6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17960960->17965055] (req 17960960->17965055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.640951:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d46c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508237440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56418us (56771us total) trans 4294971705 rc 0/0 00000100:00100000:0.0:1789760702.640961:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4417 00000100:00100000:0.0:1789760702.640970:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681628607993 time 681689229583 state 0x2 ref 0 00000100:00100000:0.0:1789760702.644606:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508237568 00000100:00100000:0.0:1789760702.644616:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508237568 00000100:00100000:0.0:1789760702.644670:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681628607993 time 681689229583 state 0x0 ref 1 00000100:00100000:0.0:1789760702.644684:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681689489999 time 681689229583 state 0x1 ref 1 00000100:00100000:0.0:1789760702.644712:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681689489999 time 681749422585 state 0x0 ref 1 00000100:00100000:0.0:1789760702.644732:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508237568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760702.644782:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17965056, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9007 00010000:00010000:0.0:1789760702.644791:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53efd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.644807:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760702.644812:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.644818:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53efd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.644832:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53efd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760702.675487:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17965056 -> 17969152 00010000:00010000:0.0:1789760702.675504:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53efd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.675517:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53efd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.675532:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53efd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.675539:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760702.675544:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53efd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.675553:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53efd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.675574:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53efd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.675583:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53efd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17965056->17969151] (req 17965056->17969151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.675724:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508237568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30990us (31199us total) trans 4294971706 rc 0/0 00000100:00100000:0.0:1789760702.675735:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4418 00000100:00100000:0.0:1789760702.675745:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681689489999 time 681749422585 state 0x2 ref 0 00000100:00100000:0.0:1789760702.679962:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508237696 00000100:00100000:0.0:1789760702.679972:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508237696 00000100:00100000:0.0:1789760702.680025:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681689489999 time 681749422585 state 0x0 ref 1 00000100:00100000:0.0:1789760702.680040:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681749683001 time 681749422585 state 0x1 ref 1 00000100:00100000:0.0:1789760702.680069:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681749683001 time 681784779799 state 0x0 ref 1 00000100:00100000:0.0:1789760702.680082:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508237696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760702.680128:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17969152, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9007 00010000:00010000:0.0:1789760702.680136:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53f04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.680150:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760702.680154:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.680161:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53f04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17969152->17973247] (req 17969152->17973247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.680174:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53f04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17969152->17973247] (req 17969152->17973247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.768197:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508237824 00000100:00100000:0.0:1789760702.768213:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508237824 00000100:00100000:0.0:1789760702.768300:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681749683001 time 681784779799 state 0x0 ref 1 00000100:00100000:0.0:1789760702.768314:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681785040215 time 681784779799 state 0x1 ref 1 00000100:00100000:0.0:1789760702.768336:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681785040215 time 681873047096 state 0x0 ref 1 00000100:00100000:0.0:1789760702.768347:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508237824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760702.768406:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17973248, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9007 00010000:00010000:0.0:1789760702.768416:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53f0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.768430:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760702.768434:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.768441:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53f0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.768454:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53f0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760702.832817:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 17973248 -> 17977344 00010000:00010000:0.0:1789760702.832834:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53f0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.832847:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53f0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.832856:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53f0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.832863:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760702.832867:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53f0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.832937:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53f0b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.832961:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53f0b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.832969:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe53f0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17973248->17977343] (req 17973248->17977343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760702.833066:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508237824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 64715us (73651us total) trans 4294971708 rc 0/0 00000100:00100000:0.0:1789760702.833078:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4420 00000100:00100000:0.0:1789760702.833096:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681785040215 time 681873047096 state 0x2 ref 0 00000100:00100000:0.0:1789760702.838421:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508237952 00000100:00100000:0.0:1789760702.838431:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508237952 00000100:00100000:0.0:1789760702.838488:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681785040215 time 681873047096 state 0x0 ref 1 00000100:00100000:0.0:1789760702.838505:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681873307512 time 681873047096 state 0x1 ref 1 00000100:00100000:0.0:1789760702.838526:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681873307512 time 681943237053 state 0x0 ref 1 00000100:00100000:0.0:1789760702.838538:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508237952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760702.838592:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17977344, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9007 00010000:00010000:0.0:1789760702.838601:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53f12 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.838617:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=681s v=(0 0) 00010000:00010000:0.0:1789760702.838622:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.838628:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53f12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17977344->17981439] (req 17977344->17981439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.838641:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe53f12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17977344->17981439] (req 17977344->17981439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760702.912623:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971709 is committed 00000001:00080000:0.0:1789760702.950530:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971710 is committed 00000100:00100000:0.0:1789760702.971010:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508238208 00000100:00100000:0.0:1789760702.971022:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508238208 00000100:00100000:0.0:1789760702.971074:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 681943497469 time 682021775559 state 0x0 ref 1 00000100:00100000:0.0:1789760702.971088:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 682022035975 time 682021775559 state 0x1 ref 1 00000100:00100000:0.0:1789760702.971118:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 682022035975 time 682075829086 state 0x0 ref 1 00000100:00100000:0.0:1789760702.971131:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508238208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760702.971178:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17985536, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9007 00010000:00010000:0.0:1789760702.971187:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53f20 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.971203:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:0.0:1789760702.971207:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760702.971213:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53f20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17985536->17989631] (req 17985536->17989631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760702.971226:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53f20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17985536->17989631] (req 17985536->17989631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.070886:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760703.227932:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508238592 00000100:00100000:0.0:1789760703.227942:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508238592 00000100:00100000:0.0:1789760703.227995:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 682176090888 time 682259878120 state 0x0 ref 1 00000100:00100000:0.0:1789760703.228010:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 682260138536 time 682259878120 state 0x1 ref 1 00000100:00100000:0.0:1789760703.228042:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 682260138536 time 682332752461 state 0x0 ref 1 00000100:00100000:0.0:1789760703.228054:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508238592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760703.228106:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 17997824, mtime: 0x6aad94bf, atime: 0x0, ctime: 0x6aad94bf, blocks: 0x9007 00010000:00010000:0.0:1789760703.228115:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53f35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.228131:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=682s v=(0 0) 00010000:00010000:0.0:1789760703.228136:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760703.228141:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53f35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17997824->18001919] (req 17997824->18001919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760703.228154:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe53f35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [17997824->18001919] (req 17997824->18001919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760703.291826:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760703.360693:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534052864 00000100:00100000:0.0:1789760703.360702:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534052864 00000100:00100000:0.0:1789760703.360766:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:0.0:1789760703.360776:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534052864:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760703.360812:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789760703.360873:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534052864:12345-0@lo:41:osp-pre-1-0.0 Request processed in 98us (790us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760703.360880:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:0.0:1789760703.423420:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508238976 00000100:00100000:0.0:1789760703.423429:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508238976 00000100:00100000:0.0:1789760703.423481:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 682398675679 time 682477511080 state 0x0 ref 1 00000100:00100000:0.0:1789760703.423497:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 682477771496 time 682477511080 state 0x1 ref 1 00000100:00100000:0.0:1789760703.460435:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760703.732490:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760704.165317:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971726 is committed 00000100:00100000:0.0:1789760704.168526:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760704.198814:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508240384 00000100:00100000:0.0:1789760704.198824:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508240384 00000100:00100000:0.0:1789760704.198877:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683241825032 time 683273410127 state 0x0 ref 1 00000100:00100000:0.0:1789760704.198892:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683273670543 time 683273410127 state 0x1 ref 1 00000100:00100000:0.0:1789760704.198914:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683273670543 time 683303625110 state 0x0 ref 1 00000100:00100000:0.0:1789760704.198924:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508240384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760704.198969:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18055168, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9007 00010000:00010000:0.0:1789760704.198977:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53f97 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.198993:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760704.198997:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.199003:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53f97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18055168->18059263] (req 18055168->18059263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.199017:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53f97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18055168->18059263] (req 18055168->18059263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.281190:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971729 is committed 00000100:00100000:0.0:1789760704.320213:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508240768 00000100:00100000:0.0:1789760704.320223:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508240768 00000100:00100000:0.0:1789760704.320527:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683356007842 time 683390363110 state 0x0 ref 1 00000100:00100000:0.0:1789760704.320543:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683390623526 time 683390363110 state 0x1 ref 1 00000100:00100000:0.0:1789760704.320573:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683390623526 time 683425284275 state 0x0 ref 1 00000100:00100000:0.0:1789760704.320586:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508240768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760704.320631:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18067456, mtime: 0x6aad94c0, atime: 0x0, ctime: 0x6aad94c0, blocks: 0x9007 00010000:00010000:0.0:1789760704.320638:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53fac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.320651:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760704.320654:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.320658:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53fac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18067456->18071551] (req 18067456->18071551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.320668:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe53fac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18067456->18071551] (req 18067456->18071551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.381236:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971731 is committed 00000100:00100000:0.0:1789760704.381676:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508240768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61080us (61766us total) trans 4294971731 rc 0/0 00000100:00100000:0.0:1789760704.381688:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4443 00000100:00100000:0.0:1789760704.381712:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683390623526 time 683425284275 state 0x2 ref 0 00000100:00100000:0.0:1789760704.385938:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760704.412064:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971732 is committed 00000001:00080000:0.0:1789760704.438147:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971733 is committed 00000100:00100000:0.0:1789760704.440680:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508241152 00000100:00100000:0.0:1789760704.440690:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508241152 00000100:00100000:0.0:1789760704.440743:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683491129066 time 683523334738 state 0x0 ref 1 00000100:00100000:0.0:1789760704.440764:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683523595154 time 683523334738 state 0x1 ref 1 00010000:00010000:2.0:1789760704.503259:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53fcf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.503447:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760704.503451:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.503458:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53fcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.503472:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53fcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18087936->18092031] (req 18087936->18092031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760704.530972:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971736 is committed 00000100:00100000:2.0:1789760704.535493:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760704.535568:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508241536 00000100:00100000:0.0:1789760704.535579:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508241536 00000100:00100000:0.0:1789760704.535636:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683575227091 time 683607892178 state 0x0 ref 1 00000100:00100000:0.0:1789760704.535651:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683608152594 time 683607892178 state 0x1 ref 1 00000100:00100000:0.0:1789760704.535685:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683608152594 time 683640395920 state 0x0 ref 1 00000100:00100000:0.0:1789760704.535697:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508241536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760704.535745:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18092032, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9007 00010000:00010000:0.0:1789760704.535754:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53fd6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.535770:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760704.535774:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.535781:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53fd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.535794:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53fd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.556325:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18092032 -> 18096128 00010000:00010000:2.0:1789760704.556341:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53fd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.556365:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53fd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.556376:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53fd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.556388:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760704.556394:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53fd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.556404:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53fd6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.556430:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53fd6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.556438:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe53fd6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18092032->18096127] (req 18092032->18096127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.556527:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508241536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20826us (21039us total) trans 4294971737 rc 0/0 00000100:00100000:2.0:1789760704.556540:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4449 00000100:00100000:2.0:1789760704.556550:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683608152594 time 683640395920 state 0x2 ref 0 00000100:00100000:2.0:1789760704.559752:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760704.577989:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760704.578022:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760704.578044:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760704.578055:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760704.578069:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760704.578090:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508241792 00000100:00100000:0.0:1789760704.578100:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508241792 00000100:00100000:0.0:1789760704.578148:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 457 00000100:00100000:0.0:1789760704.578160:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04989033c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2408:x1876699508241792:12345-192.168.203.23@tcp:400:kworker.0 00000100:00080000:2.0:1789760704.578219:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760704.578234:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04989033c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2408:x1876699508241792:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 76us (209us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760704.578243:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 457 00000100:00080000:2.0:1789760704.579831:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760704.579885:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 819 00000100:00100000:0.0:1789760704.579902:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2409:x1876699508241920:12345-192.168.203.23@tcp:400:kworker.0 00000100:00080000:2.0:1789760704.579973:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760704.579983:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2409:x1876699508241920:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 88us (2000us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760704.579995:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 819 00000100:00100000:0.0:1789760704.580018:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 822 00000100:00100000:0.0:1789760704.580027:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508242176:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760704.580047:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2de680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508242176:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 21us (1978us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760704.580053:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 822 00002000:00010000:2.0:1789760704.582060:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18096128 -> 18100224 00010000:00010000:2.0:1789760704.582072:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe53fdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.582081:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe53fdd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.582091:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe53fdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.582096:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760704.582099:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe53fdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.582166:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe53fdd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.582182:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe53fdd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.582187:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe53fdd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18096128->18100223] (req 18096128->18100223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.582260:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508241664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22336us (22511us total) trans 4294971738 rc 0/0 00000100:00100000:2.0:1789760704.582269:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4450 00000100:00100000:2.0:1789760704.582278:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683640656336 time 683664616135 state 0x2 ref 0 00000100:00100000:0.0:1789760704.584919:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:2.0:1789760704.601172:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18100224 -> 18104320 00010000:00010000:2.0:1789760704.601186:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe53fe4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.601199:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe53fe4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.601205:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe53fe4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.601209:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760704.601212:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe53fe4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.601219:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe53fe4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.601234:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe53fe4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.601239:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe53fe4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18100224->18104319] (req 18100224->18104319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.601336:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508242432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16250us (16419us total) trans 4294971739 rc 0/0 00000100:00100000:2.0:1789760704.601358:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4451 00000100:00100000:2.0:1789760704.601368:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683664876551 time 683689782303 state 0x2 ref 0 00000100:00100000:2.0:1789760704.605316:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508242560 00000100:00100000:2.0:1789760704.605327:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508242560 00000100:00100000:2.0:1789760704.605394:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683664876551 time 683689782303 state 0x0 ref 1 00000100:00100000:2.0:1789760704.605410:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683690042719 time 683689782303 state 0x1 ref 1 00000100:00100000:2.0:1789760704.605433:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683690042719 time 683710144073 state 0x0 ref 1 00000100:00100000:2.0:1789760704.605446:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508242560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760704.605496:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18104320, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9007 00010000:00010000:2.0:1789760704.605505:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53feb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.605520:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760704.605525:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.605531:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53feb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.605545:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53feb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.627651:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18104320 -> 18108416 00010000:00010000:2.0:1789760704.627666:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53feb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.627677:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53feb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.627691:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53feb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.627755:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760704.627760:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53feb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.627769:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53feb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.627788:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53feb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.627796:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53feb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18104320->18108415] (req 18104320->18108415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.627931:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508242560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22482us (23300us total) trans 4294971740 rc 0/0 00000100:00100000:2.0:1789760704.627942:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4452 00000100:00100000:2.0:1789760704.627952:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683690042719 time 683710144073 state 0x2 ref 0 00000100:00100000:2.0:1789760704.633334:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508242688 00000100:00100000:2.0:1789760704.633345:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508242688 00000100:00100000:2.0:1789760704.633410:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683690042719 time 683710144073 state 0x0 ref 1 00000100:00100000:2.0:1789760704.633426:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683710404489 time 683710144073 state 0x1 ref 1 00000100:00100000:2.0:1789760704.633459:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683710404489 time 683738169973 state 0x0 ref 1 00000100:00100000:2.0:1789760704.633471:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508242688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760704.633520:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18108416, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9007 00010000:00010000:2.0:1789760704.633530:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53ff2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.633544:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760704.633548:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.633555:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53ff2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.633568:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe53ff2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18108416->18112511] (req 18108416->18112511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.667801:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508242816 00000100:00100000:2.0:1789760704.667812:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508242816 00000100:00100000:2.0:1789760704.667863:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683710404489 time 683738169973 state 0x0 ref 1 00000100:00100000:2.0:1789760704.667879:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683738430389 time 683738169973 state 0x1 ref 1 00000100:00100000:2.0:1789760704.667950:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683738430389 time 683772660017 state 0x0 ref 1 00000100:00100000:2.0:1789760704.667963:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508242816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760704.668009:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18112512, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9007 00010000:00010000:2.0:1789760704.668018:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53ff9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.668031:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760704.668036:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.668043:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53ff9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.668055:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe53ff9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18112512->18116607] (req 18112512->18116607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760704.717319:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971742 is committed 00000100:00100000:0.0:1789760704.721948:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760704.722043:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508242944 00000100:00100000:2.0:1789760704.722053:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508242944 00000100:00100000:2.0:1789760704.722172:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683738430389 time 683772660017 state 0x0 ref 1 00000100:00100000:2.0:1789760704.722190:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683772920433 time 683772660017 state 0x1 ref 1 00000100:00100000:2.0:1789760704.722220:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683772920433 time 683826929963 state 0x0 ref 1 00000100:00100000:2.0:1789760704.722232:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508242944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760704.722547:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18116608, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9007 00010000:00010000:0.0:1789760704.722558:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54000 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.722574:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:0.0:1789760704.722579:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760704.722585:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54000 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.722598:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54000 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760704.752618:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18116608 -> 18120704 00010000:00010000:0.0:1789760704.752637:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54000 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.752651:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54000 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.752660:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54000 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.752668:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760704.752673:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54000 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.752682:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54000 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.752714:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54000 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760704.752723:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54000 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18116608->18120703] (req 18116608->18120703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760704.752816:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508242944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30580us (30875us total) trans 4294971743 rc 0/0 00000100:00100000:0.0:1789760704.752826:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4455 00000100:00100000:0.0:1789760704.752836:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683772920433 time 683826929963 state 0x2 ref 0 00000001:00080000:0.0:1789760704.752875:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971743 is committed 00000001:00080000:0.0:1789760704.798016:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971744 is committed 00000100:00100000:0.0:1789760704.813367:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760704.866682:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760704.866997:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508243328 00000100:00100000:2.0:1789760704.867006:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508243328 00000100:00100000:2.0:1789760704.867116:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683861597903 time 683918386381 state 0x0 ref 1 00000100:00100000:2.0:1789760704.867131:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683918646797 time 683918386381 state 0x1 ref 1 00000100:00100000:2.0:1789760704.867201:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683918646797 time 683971910445 state 0x0 ref 1 00000100:00100000:2.0:1789760704.867218:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508243328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760704.867272:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18128896, mtime: 0x6aad94c1, atime: 0x0, ctime: 0x6aad94c1, blocks: 0x9007 00010000:00010000:2.0:1789760704.867282:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54015 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.867326:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=683s v=(0 0) 00010000:00010000:2.0:1789760704.867331:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760704.867337:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54015 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.867359:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54015 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760704.958514:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18128896 -> 18132992 00010000:00010000:2.0:1789760704.958530:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54015 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.958544:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54015 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.958553:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54015 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.958559:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760704.958564:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54015 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.958574:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54015 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.958596:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54015 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760704.958604:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54015 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18128896->18132991] (req 18128896->18132991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760704.958691:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508243328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 91470us (92014us total) trans 4294971746 rc 0/0 00000100:00100000:2.0:1789760704.958713:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4458 00000100:00100000:2.0:1789760704.958724:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 683918646797 time 683971910445 state 0x2 ref 0 00000100:00080000:2.0:1789760705.057691:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760705.173559:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971748 is committed 00000100:00100000:0.0:1789760705.177943:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760705.245607:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971749 is committed 00000100:00100000:0.0:1789760705.250590:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00010000:00010000:0.0:1789760705.278488:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760705.406725:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04886363100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534053376:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.406791:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760705.406846:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd9ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534053888:0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.406949:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3473:1876699534052992:0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.406985:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760705.407023:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04886362680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534053504:0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.407049:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760705.407069:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd99180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534054016:0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.407089:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760705.407106:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b7fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534054528:0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.407124:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760705.407162:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534053504 00000100:00100000:2.0:1789760705.407169:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534053504 00000100:00100000:2.0:1789760705.407214:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:2.0:1789760705.407223:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534053504:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.407320:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534053504:12345-0@lo:400:kworker.0 Request processed in 98us (269us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760705.407328:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:2.0:1789760705.407361:0:8262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534054016 00000100:00100000:2.0:1789760705.407365:0:8262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534054016 00000100:00100000:2.0:1789760705.407397:0:8262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534054528 00000100:00100000:2.0:1789760705.407400:0:8262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534054528 00000100:00100000:2.0:1789760705.407421:0:8262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:2.0:1789760705.407428:0:8262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534054016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.407460:0:8262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534054016:12345-0@lo:400:kworker.0 Request processed in 32us (369us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760705.407466:0:8262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:2.0:1789760705.407478:0:8262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:2.0:1789760705.407484:0:8262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534054528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.407508:0:8262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534054528:12345-0@lo:400:kworker.0 Request processed in 25us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760705.407514:0:8262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00080000:2.0:1789760705.407535:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760705.407539:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760705.407563:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04886362680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534053504:0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.407572:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd99180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534054016:0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.407580:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b7fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534054528:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.407970:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760705.407987:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b7fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534054400:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.408107:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760705.408140:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534053120:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.408168:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760705.408187:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04886362300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534053632:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.408208:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760705.408226:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd99c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534054144:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.408426:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760705.408448:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1789760705.408455:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534053248:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.408478:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760705.408492:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04886360380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534053760:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.408513:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760705.408531:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd9b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534054272:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.408550:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760705.408574:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534053376 00000100:00100000:0.0:1789760705.408580:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534053376 00000100:00100000:0.0:1789760705.408604:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534053120 00000100:00100000:0.0:1789760705.408607:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534053120 00000100:00100000:0.0:1789760705.408618:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534053632 00000100:00100000:0.0:1789760705.408621:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534053632 00000100:00100000:0.0:1789760705.408638:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534054144 00000100:00100000:0.0:1789760705.408640:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534054144 00000100:00100000:0.0:1789760705.408653:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534053760 00000100:00100000:0.0:1789760705.408655:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534053760 00000100:00100000:0.0:1789760705.408668:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534054272 00000100:00100000:0.0:1789760705.408670:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534054272 00000100:00100000:0.0:1789760705.408685:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 823 00000100:00100000:0.0:1789760705.408692:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534053376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.408732:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534053376:12345-0@lo:400:kworker.0 Request processed in 41us (1941us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760705.408739:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 823 00000100:00100000:0.0:1789760705.408754:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 824 00000100:00100000:0.0:1789760705.408760:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534053120:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.408789:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534053120:12345-0@lo:400:kworker.0 Request processed in 29us (621us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760705.408795:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 824 00000100:00100000:0.0:1789760705.408807:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 825 00000100:00100000:0.0:1789760705.408812:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534053632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.408838:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534053632:12345-0@lo:400:kworker.0 Request processed in 26us (629us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760705.408844:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 825 00000100:00100000:0.0:1789760705.408855:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 826 00000100:00100000:2.0:1789760705.409096:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534053248 00000100:00100000:2.0:1789760705.409101:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534053248 00000100:00100000:2.0:1789760705.409129:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:2.0:1789760705.409136:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534053248:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.409171:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534053248:12345-0@lo:400:kworker.0 Request processed in 35us (691us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760705.409177:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:2.0:1789760705.409194:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534053888 00000100:00100000:2.0:1789760705.409197:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534053888 00000100:00100000:2.0:1789760705.409214:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534054400 00000100:00100000:2.0:1789760705.409216:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534054400 00000100:00100000:2.0:1789760705.409230:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:2.0:1789760705.409249:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534053888:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.409460:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534053888:12345-0@lo:400:kworker.0 Request processed in 223us (1491us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760705.409467:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:2.0:1789760705.409478:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:2.0:1789760705.409484:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534054400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760705.409509:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534054400:12345-0@lo:400:kworker.0 Request processed in 25us (1402us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760705.409514:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00080000:2.0:1789760705.409528:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760705.409531:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760705.409534:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760705.409536:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760705.409606:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534054144:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.409638:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534054144:12345-0@lo:400:kworker.0 Request processed in 36us (1212us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760705.409646:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 826 00000100:00100000:0.0:1789760705.409660:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 827 00000100:00100000:0.0:1789760705.409666:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534053760:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.410837:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534053760:12345-0@lo:400:kworker.0 Request processed in 25us (1176us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760705.410846:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 827 00000100:00100000:0.0:1789760705.411076:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 828 00000100:00100000:0.0:1789760705.411087:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534054272:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.411140:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534054272:12345-0@lo:400:kworker.0 Request processed in 56us (2588us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760705.411146:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 828 00000100:00100000:0.0:1789760705.411190:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04886363100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534053376:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.411207:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd9ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534053888:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.411218:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b7fe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534054400:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.411269:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534054656:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760705.412116:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760705.412152:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048846d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534053248:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.412165:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04886360380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534053760:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.412175:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd9b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534054272:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.412187:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048846d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534053120:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.412198:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04886362300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534053632:0@lo:400:kworker.0 00000100:00100000:0.0:1789760705.412208:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd99c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534054144:0@lo:400:kworker.0 00000001:00080000:0.0:1789760705.490983:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971752 is committed 00000100:00100000:0.0:1789760705.498111:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760705.540668:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971753 is committed 00000100:00100000:0.0:1789760705.547416:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760705.589806:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971754 is committed 00002000:00010000:2.0:1789760705.622473:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18165760 -> 18169856 00010000:00010000:2.0:1789760705.622489:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54054 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.622508:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54054 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.622517:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54054 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.622523:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760705.622528:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54054 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.622537:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54054 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.622554:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54054 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.622561:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54054 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18165760->18169855] (req 18165760->18169855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.622682:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508244480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31108us (31430us total) trans 4294971755 rc 0/0 00000100:00100000:2.0:1789760705.622701:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4467 00000100:00100000:2.0:1789760705.622710:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684652640780 time 684696268392 state 0x2 ref 0 00000001:00080000:0.0:1789760705.660429:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971756 is committed 00002000:00010000:2.0:1789760705.660633:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18169856 -> 18173952 00010000:00010000:2.0:1789760705.660839:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5405b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.660854:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5405b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.660864:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5405b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.660872:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760705.660876:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5405b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.660885:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5405b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.660906:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5405b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.660913:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5405b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18169856->18173951] (req 18169856->18173951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.661108:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508244608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33904us (34214us total) trans 4294971756 rc 0/0 00000100:00100000:2.0:1789760705.661118:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4468 00000100:00100000:2.0:1789760705.661128:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684696528808 time 684731896535 state 0x2 ref 0 00000100:00100000:0.0:1789760705.665453:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760705.697564:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971757 is committed 00002000:00010000:2.0:1789760705.697648:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18173952 -> 18178048 00010000:00010000:2.0:1789760705.697663:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54062 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.697677:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54062 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.697687:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54062 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.697755:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760705.697761:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54062 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.697770:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54062 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.697791:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54062 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.697798:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54062 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18173952->18178047] (req 18173952->18178047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.697918:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508244736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27389us (32664us total) trans 4294971757 rc 0/0 00000100:00100000:2.0:1789760705.697928:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4469 00000100:00100000:2.0:1789760705.697938:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684732156951 time 684775220153 state 0x2 ref 0 00000100:00100000:2.0:1789760705.702242:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508244864 00000100:00100000:2.0:1789760705.702252:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508244864 00000100:00100000:2.0:1789760705.702470:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684732156951 time 684775220153 state 0x0 ref 1 00000100:00100000:2.0:1789760705.702486:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684775480569 time 684775220153 state 0x1 ref 1 00000100:00100000:2.0:1789760705.702559:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684775480569 time 684807268108 state 0x0 ref 1 00000100:00100000:2.0:1789760705.702572:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508244864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760705.702621:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18178048, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9007 00010000:00010000:2.0:1789760705.702629:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54069 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.702644:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760705.702648:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.702654:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54069 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.702667:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54069 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760705.733421:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971758 is committed 00002000:00010000:2.0:1789760705.733537:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18178048 -> 18182144 00010000:00010000:2.0:1789760705.733554:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54069 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.733572:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54069 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.733580:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54069 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.733588:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760705.733593:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54069 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.733602:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54069 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.733621:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54069 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.733628:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54069 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18178048->18182143] (req 18178048->18182143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.733733:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508244864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31159us (31587us total) trans 4294971758 rc 0/0 00000100:00100000:2.0:1789760705.733743:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4470 00000100:00100000:2.0:1789760705.733752:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684775480569 time 684807268108 state 0x2 ref 0 00000100:00100000:2.0:1789760705.737202:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508244992 00000100:00100000:2.0:1789760705.737213:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508244992 00000100:00100000:2.0:1789760705.737388:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684775480569 time 684807268108 state 0x0 ref 1 00000100:00100000:2.0:1789760705.737403:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684807528524 time 684807268108 state 0x1 ref 1 00000100:00100000:2.0:1789760705.737468:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684807528524 time 684842177385 state 0x0 ref 1 00000100:00100000:2.0:1789760705.737480:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508244992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760705.737526:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18182144, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9007 00010000:00010000:2.0:1789760705.737534:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54070 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.737548:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760705.737552:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.737558:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54070 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.737571:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54070 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760705.762834:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971759 is committed 00002000:00010000:2.0:1789760705.762929:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18182144 -> 18186240 00010000:00010000:2.0:1789760705.762944:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54070 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.762957:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54070 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.762971:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54070 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.762978:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760705.762983:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54070 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.762993:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54070 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.763012:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54070 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.763020:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54070 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18182144->18186239] (req 18182144->18186239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.763118:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508244992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25634us (26198us total) trans 4294971759 rc 0/0 00000100:00100000:2.0:1789760705.763128:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4471 00000100:00100000:2.0:1789760705.763137:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684807528524 time 684842177385 state 0x2 ref 0 00000100:00100000:2.0:1789760705.766872:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508245120 00000100:00100000:2.0:1789760705.766890:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508245120 00000100:00100000:2.0:1789760705.766944:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684807528524 time 684842177385 state 0x0 ref 1 00000100:00100000:2.0:1789760705.766959:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684842437801 time 684842177385 state 0x1 ref 1 00000100:00100000:2.0:1789760705.766992:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684842437801 time 684871702116 state 0x0 ref 1 00000100:00100000:2.0:1789760705.767005:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508245120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760705.767054:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18186240, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9007 00010000:00010000:2.0:1789760705.767063:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54077 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.767078:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=684s v=(0 0) 00010000:00010000:2.0:1789760705.767082:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760705.767089:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54077 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18186240->18190335] (req 18186240->18190335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.767103:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54077 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18186240->18190335] (req 18186240->18190335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.813513:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760705.848755:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760705.880490:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18194432 -> 18198528 00010000:00010000:0.0:1789760705.880510:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54085 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.880525:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54085 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.880534:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54085 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.880542:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760705.880547:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54085 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.880556:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54085 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.880582:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54085 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760705.880591:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54085 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18194432->18198527] (req 18194432->18198527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760705.880698:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d45850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508245376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31261us (31938us total) trans 4294971762 rc 0/0 00000100:00100000:0.0:1789760705.880711:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4474 00000100:00100000:0.0:1789760705.880833:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684918637180 time 684954123662 state 0x2 ref 0 00000100:00100000:2.0:1789760705.884565:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:2.0:1789760705.924532:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18198528 -> 18202624 00010000:00010000:2.0:1789760705.924547:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5408c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.924560:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5408c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.924569:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5408c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.924577:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760705.924582:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5408c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.924590:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5408c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.924612:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5408c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760705.924620:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5408c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18198528->18202623] (req 18198528->18202623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760705.924723:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508245504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39782us (40161us total) trans 4294971763 rc 0/0 00000100:00100000:2.0:1789760705.924733:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4475 00000100:00100000:2.0:1789760705.924743:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 684954384078 time 684989635622 state 0x2 ref 0 00000100:00100000:2.0:1789760705.928658:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760705.994865:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971764 is committed 00000001:00080000:2.0:1789760706.043455:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971765 is committed 00000100:00100000:2.0:1789760706.049058:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508245888 00000100:00100000:2.0:1789760706.049067:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508245888 00000100:00100000:2.0:1789760706.049117:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685034080951 time 685101277383 state 0x0 ref 1 00000100:00100000:2.0:1789760706.049132:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685101537799 time 685101277383 state 0x1 ref 1 00000100:00100000:2.0:1789760706.049194:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685101537799 time 685153904387 state 0x0 ref 1 00000100:00100000:2.0:1789760706.049206:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508245888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760706.049263:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18210816, mtime: 0x6aad94c2, atime: 0x0, ctime: 0x6aad94c2, blocks: 0x9007 00010000:00010000:2.0:1789760706.049272:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe540a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.049314:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760706.049318:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.049324:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe540a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.049348:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe540a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760706.113494:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18210816 -> 18214912 00010000:00010000:2.0:1789760706.113514:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe540a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.113526:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe540a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.113533:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe540a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.113541:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760706.113546:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe540a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.113554:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe540a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.113573:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe540a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.113580:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe540a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18210816->18214911] (req 18210816->18214911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.113675:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508245888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 64465us (64819us total) trans 4294971766 rc 0/0 00000100:00100000:2.0:1789760706.113694:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4478 00000100:00100000:2.0:1789760706.113703:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685101537799 time 685153904387 state 0x2 ref 0 00000001:00080000:2.0:1789760706.113751:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971766 is committed 00000100:00100000:2.0:1789760706.117775:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760706.170618:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971767 is committed 00000001:00080000:2.0:1789760706.260198:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971768 is committed 00000100:00100000:2.0:1789760706.268487:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760706.357673:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971769 is committed 00000100:00100000:0.0:1789760706.430534:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760706.430556:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b647800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534055168:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760706.430628:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00010000:2.0:1789760706.484182:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18231296 -> 18235392 00010000:00010000:2.0:1789760706.484202:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe540c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.484216:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe540c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.484226:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe540c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.484234:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760706.484239:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe540c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.484353:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe540c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.484373:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe540c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.484381:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe540c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18231296->18235391] (req 18231296->18235391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.484472:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508246528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 80729us (80888us total) trans 4294971771 rc 0/0 00000100:00100000:2.0:1789760706.484483:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4483 00000100:00100000:2.0:1789760706.484493:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685467618349 time 685508438340 state 0x2 ref 0 00000001:00080000:2.0:1789760706.486383:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971771 is committed 00000100:00100000:2.0:1789760706.490627:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760706.549208:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18235392 -> 18239488 00002000:00010000:0.0:1789760706.549213:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760706 -> 1789760707 00002000:00010000:0.0:1789760706.549215:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760706 -> 1789760707 00010000:00010000:0.0:1789760706.549228:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe540cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.549240:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe540cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.549248:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe540cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.549255:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760706.549260:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe540cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.549267:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe540cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.549312:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe540cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760706.549320:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe540cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18235392->18239487] (req 18235392->18239487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760706.549424:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508246656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58594us (58803us total) trans 4294971772 rc 0/0 00000100:00100000:0.0:1789760706.549433:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4484 00000100:00100000:0.0:1789760706.549442:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685508698756 time 685595524577 state 0x2 ref 0 00000100:00100000:0.0:1789760706.556685:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760706.556820:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508246784 00000100:00100000:2.0:1789760706.556831:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508246784 00000100:00100000:2.0:1789760706.556898:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685508698756 time 685595524577 state 0x0 ref 1 00000100:00100000:2.0:1789760706.556913:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685595784993 time 685595524577 state 0x1 ref 1 00000100:00100000:2.0:1789760706.556937:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685595784993 time 685661648059 state 0x0 ref 1 00000100:00100000:2.0:1789760706.556950:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508246784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760706.556997:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18239488, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9007 00010000:00010000:2.0:1789760706.557006:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe540d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.557022:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760706.557026:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.557033:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe540d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18239488->18243583] (req 18239488->18243583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.557047:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe540d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18239488->18243583] (req 18239488->18243583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760706.608464:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971773 is committed 00000100:00100000:2.0:1789760706.619713:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508246912 00000100:00100000:2.0:1789760706.619722:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508246912 00000100:00100000:2.0:1789760706.619769:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685595784993 time 685661648059 state 0x0 ref 1 00000100:00100000:2.0:1789760706.619783:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685661908475 time 685661648059 state 0x1 ref 1 00000100:00100000:2.0:1789760706.619802:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685661908475 time 685724513256 state 0x0 ref 1 00000100:00100000:2.0:1789760706.619815:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508246912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760706.619859:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18243584, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9007 00010000:00010000:2.0:1789760706.619867:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe540d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.619882:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760706.619887:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.619893:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe540d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.619907:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe540d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18243584->18247679] (req 18243584->18247679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760706.719110:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971774 is committed 00000100:00100000:0.0:1789760706.725540:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:2.0:1789760706.765496:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18247680 -> 18251776 00010000:00010000:2.0:1789760706.765514:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe540e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.765528:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe540e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.765537:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe540e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.765544:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760706.765550:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe540e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.765558:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe540e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.765576:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe540e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.765583:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe540e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18247680->18251775] (req 18247680->18251775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.765676:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508247040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39929us (40141us total) trans 4294971775 rc 0/0 00000001:00080000:0.0:1789760706.765939:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971775 is committed 00000100:00100000:2.0:1789760706.767361:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4487 00000100:00100000:2.0:1789760706.767374:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685724773672 time 685830442547 state 0x2 ref 0 00000100:00100000:2.0:1789760706.771941:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760706.835693:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971776 is committed 00000100:00100000:2.0:1789760706.839836:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508247296 00000100:00100000:2.0:1789760706.839846:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508247296 00000100:00100000:2.0:1789760706.841771:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685830702963 time 685877680305 state 0x0 ref 1 00000100:00100000:2.0:1789760706.841791:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685877940721 time 685877680305 state 0x1 ref 1 00000100:00100000:2.0:1789760706.841831:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685877940721 time 685946541007 state 0x0 ref 1 00000100:00100000:2.0:1789760706.841851:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508247296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760706.841913:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18255872, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9007 00010000:00010000:2.0:1789760706.841922:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.841936:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760706.841940:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.841946:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.841959:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760706.878883:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18255872 -> 18259968 00010000:00010000:2.0:1789760706.878899:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.878913:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.878923:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.878930:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760706.878935:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.878945:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.878973:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.878981:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18255872->18259967] (req 18255872->18259967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.879380:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508247296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37532us (39632us total) trans 4294971777 rc 0/0 00000100:00100000:2.0:1789760706.879391:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4489 00000100:00100000:2.0:1789760706.879401:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685877940721 time 685946541007 state 0x2 ref 0 00000100:00100000:0.0:1789760706.889892:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760706.890448:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508247424 00000100:00100000:2.0:1789760706.890458:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508247424 00000100:00100000:2.0:1789760706.890513:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685877940721 time 685946541007 state 0x0 ref 1 00000100:00100000:2.0:1789760706.890528:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685946801423 time 685946541007 state 0x1 ref 1 00000100:00100000:2.0:1789760706.890560:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685946801423 time 685995270721 state 0x0 ref 1 00000100:00100000:2.0:1789760706.890572:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508247424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760706.890620:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18259968, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9007 00010000:00010000:2.0:1789760706.890630:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.890646:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=685s v=(0 0) 00010000:00010000:2.0:1789760706.890651:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.890658:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.890672:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18259968->18264063] (req 18259968->18264063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760706.991068:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508247552 00000100:00100000:2.0:1789760706.991079:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508247552 00000100:00100000:2.0:1789760706.991136:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685946801423 time 685995270721 state 0x0 ref 1 00000100:00100000:2.0:1789760706.991151:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685995531137 time 685995270721 state 0x1 ref 1 00000100:00100000:2.0:1789760706.991179:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685995531137 time 686095889823 state 0x0 ref 1 00000100:00100000:2.0:1789760706.991191:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508247552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760706.991525:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18264064, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9007 00010000:00010000:2.0:1789760706.991536:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.991552:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760706.991558:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760706.991564:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18264064->18268159] (req 18264064->18268159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760706.991578:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe540fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18264064->18268159] (req 18264064->18268159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.090428:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508247680 00000100:00100000:2.0:1789760707.090438:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508247680 00000100:00100000:2.0:1789760707.090512:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 685995531137 time 686095889823 state 0x0 ref 1 00000100:00100000:2.0:1789760707.090528:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686096150239 time 686095889823 state 0x1 ref 1 00000100:00100000:2.0:1789760707.090558:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686096150239 time 686195268498 state 0x0 ref 1 00000100:00100000:2.0:1789760707.090571:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508247680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760707.090621:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18268160, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9007 00010000:00010000:2.0:1789760707.090630:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54103 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.090645:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760707.090649:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.090655:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54103 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18268160->18272255] (req 18268160->18272255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.090667:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54103 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18268160->18272255] (req 18268160->18272255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760707.098469:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760707.134039:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760707.208340:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971781 is committed 00002000:00010000:0.0:1789760707.208445:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18272256 -> 18276352 00010000:00010000:0.0:1789760707.208474:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5410a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.208488:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5410a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.208498:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5410a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.208506:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760707.208511:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5410a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.208520:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5410a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.208541:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5410a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.208549:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5410a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18272256->18276351] (req 18272256->18276351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.208648:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508247808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 74411us (74613us total) trans 4294971781 rc 0/0 00000100:00100000:0.0:1789760707.208658:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4493 00000100:00100000:0.0:1789760707.208667:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686195528914 time 686238931210 state 0x2 ref 0 00000100:00100000:0.0:1789760707.218248:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:2.0:1789760707.273656:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18276352 -> 18280448 00010000:00010000:2.0:1789760707.273669:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54111 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.273697:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54111 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.273706:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54111 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.273714:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760707.273719:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54111 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.273728:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54111 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.273748:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54111 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.273755:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54111 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18276352->18280447] (req 18276352->18280447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.273850:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508247936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55370us (55609us total) trans 4294971782 rc 0/0 00000100:00100000:2.0:1789760707.273861:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4494 00000100:00100000:2.0:1789760707.273871:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686239191626 time 686323175634 state 0x2 ref 0 00000001:00080000:2.0:1789760707.316260:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971783 is committed 00000100:00100000:0.0:1789760707.321442:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00010000:00010000:0.0:1789760707.329743:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760707.387186:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508248320 00000100:00100000:2.0:1789760707.387196:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508248320 00000100:00100000:2.0:1789760707.387255:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686383294968 time 686426686605 state 0x0 ref 1 00000100:00100000:2.0:1789760707.387270:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686426947021 time 686426686605 state 0x1 ref 1 00000100:00100000:2.0:1789760707.387341:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686426947021 time 686492050680 state 0x0 ref 1 00000100:00100000:2.0:1789760707.387353:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508248320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760707.387400:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18288640, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9007 00010000:00010000:2.0:1789760707.387408:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54126 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.387423:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760707.387428:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.387434:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54126 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.387447:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54126 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760707.414110:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18288640 -> 18292736 00010000:00010000:2.0:1789760707.414131:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54126 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.414144:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54126 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.414157:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54126 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.414164:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760707.414169:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54126 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.414177:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54126 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.414196:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54126 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.414203:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54126 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18288640->18292735] (req 18288640->18292735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.415132:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508248320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27775us (28080us total) trans 4294971785 rc 0/0 00000100:00100000:2.0:1789760707.415143:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4497 00000100:00100000:2.0:1789760707.415152:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686426947021 time 686492050680 state 0x2 ref 0 00000100:00100000:2.0:1789760707.422828:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760707.494368:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508248576 00000100:00100000:0.0:1789760707.494378:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508248576 00000100:00100000:0.0:1789760707.494429:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686492311096 time 686527972235 state 0x0 ref 1 00000100:00100000:0.0:1789760707.494444:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686528232651 time 686527972235 state 0x1 ref 1 00000100:00100000:0.0:1789760707.494472:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686528232651 time 686599182257 state 0x0 ref 1 00000100:00100000:0.0:1789760707.494484:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508248576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760707.494530:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18296832, mtime: 0x6aad94c3, atime: 0x0, ctime: 0x6aad94c3, blocks: 0x9007 00010000:00010000:0.0:1789760707.494539:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54134 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.494554:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:0.0:1789760707.494558:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760707.494565:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54134 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.494578:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54134 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.495829:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971786 is committed 00000001:00080000:2.0:1789760707.578693:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971787 is committed 00002000:00010000:0.0:1789760707.578952:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18296832 -> 18300928 00002000:00010000:0.0:1789760707.578957:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760707 -> 1789760708 00002000:00010000:0.0:1789760707.578960:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760707 -> 1789760708 00010000:00010000:0.0:1789760707.578972:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54134 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.579058:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54134 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.579067:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54134 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.579074:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760707.579079:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54134 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.579087:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54134 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.579106:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54134 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760707.579113:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54134 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18296832->18300927] (req 18296832->18300927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760707.579209:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508248576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 84722us (85171us total) trans 4294971787 rc 0/0 00000100:00100000:0.0:1789760707.579219:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4499 00000100:00100000:0.0:1789760707.579228:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686528232651 time 686599182257 state 0x2 ref 0 00000100:00100000:0.0:1789760707.597128:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760707.600373:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508248704 00000100:00100000:2.0:1789760707.600389:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508248704 00000100:00100000:2.0:1789760707.600521:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686528232651 time 686599182257 state 0x0 ref 1 00000100:00100000:2.0:1789760707.600537:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686599442673 time 686599182257 state 0x1 ref 1 00000100:00100000:2.0:1789760707.600626:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686599442673 time 686705336118 state 0x0 ref 1 00000100:00100000:2.0:1789760707.600639:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508248704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760707.600684:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18300928, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9007 00010000:00010000:2.0:1789760707.600693:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5413b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.600709:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760707.600714:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.600731:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5413b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.600746:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5413b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760707.686204:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18300928 -> 18305024 00010000:00010000:2.0:1789760707.686223:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5413b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.686236:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5413b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.686250:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5413b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.686257:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760707.686262:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5413b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.686270:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5413b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.686334:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5413b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.686344:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5413b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18300928->18305023] (req 18300928->18305023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760707.686697:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508248704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 86054us (89575us total) trans 4294971788 rc 0/0 00000100:00100000:2.0:1789760707.686708:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4500 00000100:00100000:2.0:1789760707.686725:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686599442673 time 686705336118 state 0x2 ref 0 00000001:00080000:2.0:1789760707.686764:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971788 is committed 00000100:00100000:2.0:1789760707.694571:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760707.794903:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508248832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 99013us (100334us total) trans 4294971789 rc 0/0 00000100:00100000:2.0:1789760707.794918:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4501 00000100:00100000:2.0:1789760707.794929:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686705596534 time 686800582539 state 0x2 ref 0 00000100:00100000:0.0:1789760707.806276:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760707.874878:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760707.874977:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508249088 00000100:00100000:2.0:1789760707.874986:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508249088 00000100:00100000:2.0:1789760707.875040:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686800842955 time 686911324299 state 0x0 ref 1 00000100:00100000:2.0:1789760707.875055:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686911584715 time 686911324299 state 0x1 ref 1 00000100:00100000:2.0:1789760707.875086:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686911584715 time 686979796534 state 0x0 ref 1 00000100:00100000:2.0:1789760707.875097:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508249088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760707.875145:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18313216, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9007 00010000:00010000:2.0:1789760707.875154:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54150 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.875169:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=686s v=(0 0) 00010000:00010000:2.0:1789760707.875174:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760707.875180:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54150 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760707.875194:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54150 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18313216->18317311] (req 18313216->18317311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760707.950644:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971791 is committed 00000100:00100000:2.0:1789760707.971768:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760708.019171:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971792 is committed 00002000:00010000:0.0:1789760708.019345:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18317312 -> 18321408 00010000:00010000:0.0:1789760708.019361:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54157 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.019373:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54157 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.019382:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54157 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.019390:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760708.019396:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54157 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.019405:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54157 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.019427:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54157 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.019435:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54157 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18317312->18321407] (req 18317312->18321407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.019534:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508249216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47272us (47770us total) trans 4294971792 rc 0/0 00000100:00100000:0.0:1789760708.019543:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4504 00000100:00100000:0.0:1789760708.019552:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 686980056950 time 687076954598 state 0x2 ref 0 00000100:00100000:0.0:1789760708.031258:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760708.073380:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971793 is committed 00002000:00010000:2.0:1789760708.073585:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18321408 -> 18325504 00010000:00010000:2.0:1789760708.073600:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5415e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.073613:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5415e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.073622:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5415e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.073629:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760708.073634:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5415e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.073643:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5415e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.073665:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5415e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.073682:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5415e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18321408->18325503] (req 18321408->18325503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.073776:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508249344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42242us (42522us total) trans 4294971793 rc 0/0 00000100:00100000:2.0:1789760708.073787:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4505 00000100:00100000:2.0:1789760708.073797:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687077215014 time 687136228786 state 0x2 ref 0 00000100:00100000:2.0:1789760708.078246:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508249472 00000100:00100000:2.0:1789760708.078264:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508249472 00000100:00100000:2.0:1789760708.078358:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687077215014 time 687136228786 state 0x0 ref 1 00000100:00100000:2.0:1789760708.078373:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687136489202 time 687136228786 state 0x1 ref 1 00000100:00100000:2.0:1789760708.078394:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687136489202 time 687183104507 state 0x0 ref 1 00000100:00100000:2.0:1789760708.078407:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508249472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760708.078467:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18325504, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9007 00010000:00010000:2.0:1789760708.078476:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54165 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.078492:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760708.078497:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.078504:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54165 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.078518:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54165 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760708.157082:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18325504 -> 18329600 00010000:00010000:0.0:1789760708.157101:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54165 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.157114:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54165 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.157122:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54165 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.157130:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760708.157135:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54165 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.157145:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54165 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.157165:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54165 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.157174:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54165 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18325504->18329599] (req 18325504->18329599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.157447:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508249472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 79037us (79295us total) trans 4294971794 rc 0/0 00000100:00100000:0.0:1789760708.157458:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4506 00000100:00100000:0.0:1789760708.157467:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687136489202 time 687183104507 state 0x2 ref 0 00000001:00080000:0.0:1789760708.157505:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971794 is committed 00000100:00100000:0.0:1789760708.162596:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508249600 00000100:00100000:0.0:1789760708.162608:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508249600 00000100:00100000:0.0:1789760708.163074:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687136489202 time 687183104507 state 0x0 ref 1 00000100:00100000:0.0:1789760708.163093:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687183364923 time 687183104507 state 0x1 ref 1 00000100:00100000:0.0:1789760708.163129:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687183364923 time 687267838735 state 0x0 ref 1 00000100:00100000:0.0:1789760708.163143:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508249600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760708.163196:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18329600, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9007 00010000:00010000:0.0:1789760708.163204:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5416c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.163220:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:0.0:1789760708.163225:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.163242:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5416c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.163258:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5416c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760708.222034:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18329600 -> 18333696 00010000:00010000:0.0:1789760708.222049:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5416c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.222061:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5416c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.222069:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5416c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.222075:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760708.222079:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5416c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.222088:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5416c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.222107:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5416c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.222114:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5416c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18329600->18333695] (req 18329600->18333695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.222254:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508249600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59110us (60848us total) trans 4294971795 rc 0/0 00000100:00100000:0.0:1789760708.222268:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4507 00000100:00100000:0.0:1789760708.222277:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687183364923 time 687267838735 state 0x2 ref 0 00000100:00100000:2.0:1789760708.226561:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508249728 00000100:00100000:2.0:1789760708.226570:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508249728 00000100:00100000:2.0:1789760708.226622:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687183364923 time 687267838735 state 0x0 ref 1 00000100:00100000:2.0:1789760708.226637:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687268099151 time 687267838735 state 0x1 ref 1 00000100:00100000:2.0:1789760708.226669:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687268099151 time 687331379010 state 0x0 ref 1 00000100:00100000:2.0:1789760708.226689:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508249728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760708.226737:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18333696, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9007 00010000:00010000:2.0:1789760708.226746:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54173 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.226762:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760708.226767:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.226774:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54173 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.226789:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54173 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760708.254999:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18333696 -> 18337792 00010000:00010000:2.0:1789760708.255015:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54173 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.255027:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54173 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.255037:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54173 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.255044:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760708.255050:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54173 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.255059:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54173 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.255078:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54173 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.255087:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54173 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18333696->18337791] (req 18333696->18337791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.255224:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508249728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28539us (28999us total) trans 4294971796 rc 0/0 00000100:00100000:2.0:1789760708.255379:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4508 00000100:00100000:2.0:1789760708.255390:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687268099151 time 687331379010 state 0x2 ref 0 00000100:00100000:2.0:1789760708.259696:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508249856 00000100:00100000:2.0:1789760708.259705:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508249856 00000100:00100000:2.0:1789760708.259758:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687268099151 time 687331379010 state 0x0 ref 1 00000100:00100000:2.0:1789760708.259774:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687331639426 time 687331379010 state 0x1 ref 1 00000100:00100000:2.0:1789760708.259803:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687331639426 time 687364513649 state 0x0 ref 1 00000100:00100000:2.0:1789760708.259816:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508249856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760708.259861:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18337792, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9007 00010000:00010000:2.0:1789760708.259870:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5417a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.259897:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760708.259902:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.259908:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5417a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.259922:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5417a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760708.319663:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18337792 -> 18341888 00010000:00010000:2.0:1789760708.319686:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5417a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.319698:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5417a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.319706:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5417a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.319713:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760708.319718:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5417a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.319728:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5417a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.319753:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5417a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.319762:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5417a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18337792->18341887] (req 18337792->18341887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760708.319854:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508249856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60034us (60253us total) trans 4294971797 rc 0/0 00000100:00100000:2.0:1789760708.319865:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4509 00000100:00100000:2.0:1789760708.319875:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687331639426 time 687364513649 state 0x2 ref 0 00000100:00100000:0.0:1789760708.330277:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508249984 00000100:00100000:0.0:1789760708.330435:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508249984 00000100:00100000:0.0:1789760708.330498:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687331639426 time 687364513649 state 0x0 ref 1 00000100:00100000:0.0:1789760708.330513:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687364774065 time 687364513649 state 0x1 ref 1 00000100:00100000:0.0:1789760708.330535:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687364774065 time 687435246215 state 0x0 ref 1 00000100:00100000:0.0:1789760708.330548:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508249984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760708.330592:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18341888, mtime: 0x6aad94c4, atime: 0x0, ctime: 0x6aad94c4, blocks: 0x9007 00010000:00010000:0.0:1789760708.330601:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54181 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.330615:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:0.0:1789760708.330620:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.330626:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54181 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.330639:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54181 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18341888->18345983] (req 18341888->18345983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.488968:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534055296 00000100:00100000:0.0:1789760708.488976:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534055296 00000100:00100000:0.0:1789760708.489045:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:0.0:1789760708.489054:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534055296:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760708.489094:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789760708.489143:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534055296:12345-0@lo:41:osp-pre-1-0.0 Request processed in 90us (649us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760708.489150:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:0.0:1789760708.489213:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048864f8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534055296:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760708.526155:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508250112 00000100:00100000:2.0:1789760708.526164:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508250112 00000100:00100000:2.0:1789760708.526219:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687364774065 time 687435246215 state 0x0 ref 1 00000100:00100000:2.0:1789760708.526244:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687435506631 time 687435246215 state 0x1 ref 1 00000100:00100000:2.0:1789760708.526272:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687435506631 time 687630982927 state 0x0 ref 1 00000100:00100000:2.0:1789760708.526537:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508250112:12345-192.168.203.23@tcp:4:dd.0 00000100:00100000:0.0:1789760708.605645:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760708.605858:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508250240 00000100:00100000:2.0:1789760708.605868:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508250240 00000100:00100000:2.0:1789760708.605924:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687435506631 time 687630982927 state 0x0 ref 1 00000100:00100000:2.0:1789760708.605939:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687631243343 time 687630982927 state 0x1 ref 1 00000100:00100000:2.0:1789760708.605975:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687631243343 time 687710684887 state 0x0 ref 1 00000100:00100000:2.0:1789760708.605998:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508250240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760708.606049:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18350080, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9007 00010000:00010000:2.0:1789760708.606059:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5418f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.606075:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:2.0:1789760708.606080:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760708.606086:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5418f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.606100:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5418f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760708.718235:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18350080 -> 18354176 00010000:00010000:2.0:1789760708.718265:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5418f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.718278:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5418f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.718329:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5418f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.718337:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760708.718342:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5418f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.718353:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5418f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.718373:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5418f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760708.718380:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5418f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18350080->18354175] (req 18350080->18354175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760708.718596:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971800 is committed 00000100:00100000:0.0:1789760708.733371:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508250368 00000100:00100000:0.0:1789760708.733385:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508250368 00000100:00100000:0.0:1789760708.733596:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687631243343 time 687710684887 state 0x0 ref 1 00000100:00100000:0.0:1789760708.733612:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687710945303 time 687710684887 state 0x1 ref 1 00000100:00100000:0.0:1789760708.733649:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687710945303 time 687838358501 state 0x0 ref 1 00000100:00100000:0.0:1789760708.733662:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508250368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760708.733720:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18354176, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9007 00010000:00010000:0.0:1789760708.733729:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54196 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.733746:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=687s v=(0 0) 00010000:00010000:0.0:1789760708.733751:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760708.733758:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54196 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.733776:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54196 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18354176->18358271] (req 18354176->18358271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760708.819076:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971801 is committed 00000100:00100000:0.0:1789760708.830222:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760708.927698:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971802 is committed 00002000:00010000:0.0:1789760708.927823:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18358272 -> 18362368 00010000:00010000:0.0:1789760708.927838:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5419d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.927849:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5419d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.927857:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5419d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.927864:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760708.927868:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5419d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.927875:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5419d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.927895:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5419d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760708.927902:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5419d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18358272->18362367] (req 18358272->18362367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760708.928030:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508250496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 97569us (97815us total) trans 4294971802 rc 0/0 00000100:00100000:0.0:1789760708.928041:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4514 00000100:00100000:0.0:1789760708.928050:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 687838618917 time 687935155934 state 0x2 ref 0 00000100:00100000:0.0:1789760709.008985:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00080000:0.0:1789760709.010067:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789760709.037340:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe541ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.037351:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe541ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18366464->18370559] (req 18366464->18370559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.037759:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508250752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28388us (28777us total) trans 4294971804 rc 0/0 00000100:00100000:1.0:1789760709.037770:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4516 00000100:00100000:1.0:1789760709.037780:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688038163373 time 688114063801 state 0x2 ref 0 00000100:00100000:0.0:1789760709.041688:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760709.041772:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508250880 00000100:00100000:1.0:1789760709.041782:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508250880 00000100:00100000:1.0:1789760709.041832:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688038163373 time 688114063801 state 0x0 ref 1 00000100:00100000:1.0:1789760709.041849:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688114324217 time 688114063801 state 0x1 ref 1 00000100:00100000:1.0:1789760709.041883:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688114324217 time 688146593049 state 0x0 ref 1 00000100:00100000:1.0:1789760709.041895:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508250880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760709.041940:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18370560, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9007 00010000:00010000:1.0:1789760709.041949:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe541b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.041965:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:1.0:1789760709.041970:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.041976:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe541b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.041995:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe541b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.098457:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971805 is committed 00002000:00010000:1.0:1789760709.098611:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18370560 -> 18374656 00010000:00010000:1.0:1789760709.098626:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe541b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.098639:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe541b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.098648:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe541b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.098656:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760709.098661:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe541b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.098681:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe541b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.098700:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe541b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.098707:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe541b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18370560->18374655] (req 18370560->18374655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.098813:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508250880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56913us (57127us total) trans 4294971805 rc 0/0 00000100:00100000:1.0:1789760709.098824:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4517 00000100:00100000:1.0:1789760709.098834:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688114324217 time 688146593049 state 0x2 ref 0 00000100:00100000:2.0:1789760709.115211:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760709.122434:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688114324217 time 688146593049 state 0x0 ref 1 00000100:00100000:2.0:1789760709.122453:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688146853465 time 688146593049 state 0x1 ref 1 00000100:00100000:2.0:1789760709.122546:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688146853465 time 688227256193 state 0x0 ref 1 00000100:00100000:2.0:1789760709.122560:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508251008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760709.122607:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18374656, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9007 00010000:00010000:2.0:1789760709.122616:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.122636:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760709.122640:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.122647:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.122661:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760709.225222:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18374656 -> 18378752 00010000:00010000:2.0:1789760709.225246:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.225257:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.225265:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.225272:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760709.225276:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.225764:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.225789:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.225799:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18374656->18378751] (req 18374656->18378751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760709.225888:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508251008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 103326us (110682us total) trans 4294971806 rc 0/0 00000100:00100000:2.0:1789760709.225899:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4518 00000100:00100000:2.0:1789760709.225909:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688146853465 time 688227256193 state 0x2 ref 0 00000100:00100000:1.0:1789760709.229226:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508251136 00000100:00100000:1.0:1789760709.229246:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508251136 00000100:00100000:1.0:1789760709.229351:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688146853465 time 688227256193 state 0x0 ref 1 00000100:00100000:1.0:1789760709.229366:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688227516609 time 688227256193 state 0x1 ref 1 00000100:00100000:1.0:1789760709.229392:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688227516609 time 688334102888 state 0x0 ref 1 00000100:00100000:1.0:1789760709.229404:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508251136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760709.229450:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18378752, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9007 00010000:00010000:1.0:1789760709.229458:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe541c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.229472:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:1.0:1789760709.229477:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.229482:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe541c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18378752->18382847] (req 18378752->18382847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.229497:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe541c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18378752->18382847] (req 18378752->18382847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.294057:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971807 is committed 00000100:00100000:1.0:1789760709.310733:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760709.322689:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508251264 00000100:00100000:0.0:1789760709.322704:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508251264 00000100:00100000:0.0:1789760709.322768:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688227516609 time 688334102888 state 0x0 ref 1 00000100:00100000:0.0:1789760709.322783:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688334363304 time 688334102888 state 0x1 ref 1 00000100:00100000:0.0:1789760709.322807:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688334363304 time 688427517431 state 0x0 ref 1 00000100:00100000:0.0:1789760709.322820:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508251264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760709.322866:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18382848, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9007 00010000:00010000:0.0:1789760709.322875:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.322890:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:0.0:1789760709.322894:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.322900:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.322913:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760709.373716:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971808 is committed 00002000:00010000:0.0:1789760709.373775:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18382848 -> 18386944 00010000:00010000:0.0:1789760709.373791:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.373803:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.373813:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.373821:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760709.373826:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.373835:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.373856:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.373865:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18382848->18386943] (req 18382848->18386943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.373958:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508251264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51136us (63231us total) trans 4294971808 rc 0/0 00000100:00100000:0.0:1789760709.373968:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4520 00000100:00100000:0.0:1789760709.373978:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688334363304 time 688427517431 state 0x2 ref 0 00000100:00100000:1.0:1789760709.382782:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760709.382916:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508251392 00000100:00100000:2.0:1789760709.382925:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508251392 00000100:00100000:2.0:1789760709.382977:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688334363304 time 688427517431 state 0x0 ref 1 00000100:00100000:2.0:1789760709.382991:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688427777847 time 688427517431 state 0x1 ref 1 00000100:00100000:2.0:1789760709.383025:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688427777847 time 688487735639 state 0x0 ref 1 00000100:00100000:2.0:1789760709.383037:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508251392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760709.383084:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18386944, mtime: 0x6aad94c5, atime: 0x0, ctime: 0x6aad94c5, blocks: 0x9007 00010000:00010000:2.0:1789760709.383093:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.383109:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760709.383114:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.383120:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.383139:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760709.482548:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971809 is committed 00002000:00010000:1.0:1789760709.484686:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18386944 -> 18391040 00010000:00010000:1.0:1789760709.484703:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.484715:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.484723:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.484729:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760709.484734:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.484742:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.484760:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.484767:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe541ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18386944->18391039] (req 18386944->18391039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760709.484886:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508251392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 101837us (102102us total) trans 4294971809 rc 0/0 00000100:00100000:1.0:1789760709.484896:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4521 00000100:00100000:1.0:1789760709.484906:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688427777847 time 688487735639 state 0x2 ref 0 00000100:00100000:0.0:1789760709.492426:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760709.651844:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971810 is committed 00002000:00010000:0.0:1789760709.652083:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18391040 -> 18395136 00002000:00010000:0.0:1789760709.652087:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760709 -> 1789760710 00002000:00010000:0.0:1789760709.652090:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760709 -> 1789760710 00010000:00010000:0.0:1789760709.652104:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe541d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.652117:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe541d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.652127:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe541d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.652135:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760709.652140:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe541d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.652149:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe541d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.652175:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe541d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.652184:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe541d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18391040->18395135] (req 18391040->18395135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.652279:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508251520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 158754us (159868us total) trans 4294971810 rc 0/0 00000100:00100000:0.0:1789760709.652335:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4522 00000100:00100000:0.0:1789760709.652345:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688487996055 time 688598220243 state 0x2 ref 0 00000100:00100000:0.0:1789760709.662603:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760709.667791:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508251648 00000100:00100000:2.0:1789760709.667805:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508251648 00000100:00100000:2.0:1789760709.667867:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688487996055 time 688598220243 state 0x0 ref 1 00000100:00100000:2.0:1789760709.667883:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688598480659 time 688598220243 state 0x1 ref 1 00000100:00100000:2.0:1789760709.667912:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688598480659 time 688772622790 state 0x0 ref 1 00000100:00100000:2.0:1789760709.667925:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508251648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760709.667975:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18395136, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9007 00010000:00010000:2.0:1789760709.667983:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe541dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.667997:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:2.0:1789760709.668001:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760709.668007:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe541dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760709.668021:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe541dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.699545:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760709.699588:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760709.699612:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760709.699631:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760709.699649:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760709.699672:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508251776 00000100:00100000:0.0:1789760709.699681:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508251776 00000100:00100000:0.0:1789760709.699725:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 459 00000100:00100000:0.0:1789760709.699734:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04989033050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2407:x1876699508251776:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760709.699804:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04989033050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2407:x1876699508251776:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 71us (189us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760709.699811:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 459 00000100:00080000:0.0:1789760709.701269:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789760709.701274:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760709.701354:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 831 00000100:00100000:0.0:1789760709.701365:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508252032:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760709.701402:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508252032:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 39us (1768us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760709.701408:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 831 00000100:00100000:0.0:1789760709.701423:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 832 00000100:00100000:0.0:1789760709.701429:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508252288:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760709.701452:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508252288:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 23us (1802us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760709.701458:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 832 00000001:00080000:2.0:1789760709.794006:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971811 is committed 00002000:00010000:0.0:1789760709.794236:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18395136 -> 18399232 00010000:00010000:0.0:1789760709.794251:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe541dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.794262:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe541dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.794270:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe541dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.794277:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760709.794282:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe541dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.794330:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe541dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.794351:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe541dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.794359:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe541dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18395136->18399231] (req 18395136->18399231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.794561:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508251648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 126632us (131961us total) trans 4294971811 rc 0/0 00000100:00100000:0.0:1789760709.794572:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4523 00000100:00100000:0.0:1789760709.794583:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688598480659 time 688772622790 state 0x2 ref 0 00000100:00100000:0.0:1789760709.809701:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760709.810039:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508252416 00000100:00100000:1.0:1789760709.810050:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508252416 00000100:00100000:1.0:1789760709.810104:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688598480659 time 688772622790 state 0x0 ref 1 00000100:00100000:1.0:1789760709.810122:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688772883206 time 688772622790 state 0x1 ref 1 00000100:00100000:1.0:1789760709.810160:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688772883206 time 688914869889 state 0x0 ref 1 00000100:00100000:1.0:1789760709.810173:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508252416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760709.810223:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18399232, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9007 00010000:00010000:1.0:1789760709.810233:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe541e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.810247:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=688s v=(0 0) 00010000:00010000:1.0:1789760709.810250:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760709.810256:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe541e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760709.810269:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe541e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760709.906726:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971812 is committed 00002000:00010000:0.0:1789760709.908123:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18399232 -> 18403328 00010000:00010000:0.0:1789760709.908139:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe541e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.908152:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe541e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.908162:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe541e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.908170:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760709.908175:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe541e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.908190:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe541e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.908212:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe541e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.908221:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe541e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18399232->18403327] (req 18399232->18403327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760709.908368:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508252416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 98192us (98671us total) trans 4294971812 rc 0/0 00000100:00100000:0.0:1789760709.908378:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4524 00000100:00100000:0.0:1789760709.908387:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688772883206 time 688914869889 state 0x2 ref 0 00000100:00100000:1.0:1789760709.913542:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760709.915097:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508252544 00000100:00100000:0.0:1789760709.915110:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508252544 00000100:00100000:0.0:1789760709.915169:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688772883206 time 688914869889 state 0x0 ref 1 00000100:00100000:0.0:1789760709.915183:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688915130305 time 688914869889 state 0x1 ref 1 00000100:00100000:0.0:1789760709.915218:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688915130305 time 689019929063 state 0x0 ref 1 00000100:00100000:0.0:1789760709.915232:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508252544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760709.915297:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18403328, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9007 00010000:00010000:0.0:1789760709.915306:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.915331:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760709.915336:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760709.915342:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760709.915355:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.027918:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971813 is committed 00002000:00010000:0.0:1789760710.028130:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18403328 -> 18407424 00010000:00010000:0.0:1789760710.028145:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.028156:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.028164:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.028171:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760710.028175:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.028184:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.028204:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.028211:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18403328->18407423] (req 18403328->18407423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.028359:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508252544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 113124us (114820us total) trans 4294971813 rc 0/0 00000100:00100000:0.0:1789760710.028369:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4525 00000100:00100000:0.0:1789760710.028379:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688915130305 time 689019929063 state 0x2 ref 0 00000100:00100000:2.0:1789760710.049672:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760710.050349:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508252672 00000100:00100000:0.0:1789760710.050360:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508252672 00000100:00100000:0.0:1789760710.050428:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 688915130305 time 689019929063 state 0x0 ref 1 00000100:00100000:0.0:1789760710.050445:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689020189479 time 689019929063 state 0x1 ref 1 00000100:00100000:0.0:1789760710.050477:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689020189479 time 689155187639 state 0x0 ref 1 00000100:00100000:0.0:1789760710.050491:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508252672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760710.050546:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18407424, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9007 00010000:00010000:0.0:1789760710.050555:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.050572:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760710.050577:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.050584:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.050606:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.149627:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971814 is committed 00002000:00010000:0.0:1789760710.149905:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18407424 -> 18411520 00010000:00010000:0.0:1789760710.149920:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.149932:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.149940:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.149946:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760710.149951:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.149958:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.149977:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.149983:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18407424->18411519] (req 18407424->18411519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.150085:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508252672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 99591us (100418us total) trans 4294971814 rc 0/0 00000100:00100000:0.0:1789760710.150096:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4526 00000100:00100000:0.0:1789760710.150105:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689020189479 time 689155187639 state 0x2 ref 0 00000100:00100000:1.0:1789760710.159590:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760710.159789:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508252800 00000100:00100000:0.0:1789760710.159799:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508252800 00000100:00100000:0.0:1789760710.159855:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689020189479 time 689155187639 state 0x0 ref 1 00000100:00100000:0.0:1789760710.159870:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689155448055 time 689155187639 state 0x1 ref 1 00000100:00100000:0.0:1789760710.159901:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689155448055 time 689264611293 state 0x0 ref 1 00000100:00100000:0.0:1789760710.159913:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508252800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760710.159958:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18411520, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9007 00010000:00010000:0.0:1789760710.159966:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.159982:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760710.159986:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.159992:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.160005:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.288002:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971815 is committed 00002000:00010000:2.0:1789760710.288066:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18411520 -> 18415616 00010000:00010000:2.0:1789760710.288081:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.288093:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.288102:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.288109:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760710.288113:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.288122:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.288138:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.288150:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe541f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18411520->18415615] (req 18411520->18415615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.288243:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508252800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 128327us (128658us total) trans 4294971815 rc 0/0 00000100:00100000:2.0:1789760710.288253:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4527 00000100:00100000:2.0:1789760710.288262:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689155448055 time 689264611293 state 0x2 ref 0 00000100:00100000:2.0:1789760710.308404:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760710.308490:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508252928 00000100:00100000:0.0:1789760710.308501:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508252928 00000100:00100000:0.0:1789760710.308556:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689155448055 time 689264611293 state 0x0 ref 1 00000100:00100000:0.0:1789760710.308571:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689264871709 time 689264611293 state 0x1 ref 1 00000100:00100000:0.0:1789760710.308602:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689264871709 time 689413312552 state 0x0 ref 1 00000100:00100000:0.0:1789760710.308615:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508252928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760710.308675:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18415616, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9007 00010000:00010000:0.0:1789760710.308684:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe541ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.308697:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760710.308701:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.308707:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe541ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.308722:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe541ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760710.367067:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18415616 -> 18419712 00010000:00010000:0.0:1789760710.367086:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe541ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.367104:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe541ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.367113:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe541ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.367121:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760710.367126:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe541ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.367135:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe541ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.367154:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe541ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.367162:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe541ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18415616->18419711] (req 18415616->18419711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760710.367456:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971816 is committed 00000100:00100000:0.0:1789760710.367593:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508252928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58974us (59195us total) trans 4294971816 rc 0/0 00000100:00100000:0.0:1789760710.367604:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4528 00000100:00100000:0.0:1789760710.367613:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689264871709 time 689413312552 state 0x2 ref 0 00000100:00100000:0.0:1789760710.382173:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760710.382256:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508253056 00000100:00100000:2.0:1789760710.382266:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508253056 00000100:00100000:2.0:1789760710.382447:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689264871709 time 689413312552 state 0x0 ref 1 00000100:00100000:2.0:1789760710.382463:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689413572968 time 689413312552 state 0x1 ref 1 00000100:00100000:2.0:1789760710.382494:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689413572968 time 689487204287 state 0x0 ref 1 00000100:00100000:2.0:1789760710.382507:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508253056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760710.382557:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18419712, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9007 00010000:00010000:2.0:1789760710.382567:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54206 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.382582:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760710.382587:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.382594:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54206 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.382607:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54206 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18419712->18423807] (req 18419712->18423807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.399062:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760710.432027:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971817 is committed 00000100:00100000:0.0:1789760710.435422:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760710.483630:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971818 is committed 00002000:00010000:2.0:1789760710.487597:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18423808 -> 18427904 00010000:00010000:2.0:1789760710.487616:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5420d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.487628:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5420d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.487637:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5420d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.487644:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760710.487648:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5420d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.487712:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5420d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.487733:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5420d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.487741:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5420d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18423808->18427903] (req 18423808->18427903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.487840:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508253184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52125us (52421us total) trans 4294971818 rc 0/0 00000100:00100000:2.0:1789760710.487853:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4530 00000100:00100000:2.0:1789760710.487862:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689487464703 time 689540409599 state 0x2 ref 0 00000100:00100000:2.0:1789760710.491699:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760710.491834:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508253312 00000100:00100000:0.0:1789760710.491844:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508253312 00000100:00100000:0.0:1789760710.491899:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689487464703 time 689540409599 state 0x0 ref 1 00000100:00100000:0.0:1789760710.491916:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689540670015 time 689540409599 state 0x1 ref 1 00000100:00100000:0.0:1789760710.491948:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689540670015 time 689596658585 state 0x0 ref 1 00000100:00100000:0.0:1789760710.491961:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d44c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508253312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760710.492013:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18427904, mtime: 0x6aad94c6, atime: 0x0, ctime: 0x6aad94c6, blocks: 0x9007 00010000:00010000:0.0:1789760710.492023:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54214 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.492038:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760710.492043:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.492050:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54214 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.492065:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54214 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.533981:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b647480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534055424:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760710.534053:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.534113:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760710.534120:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b644700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534055552:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760710.534147:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.534198:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534055424 00000100:00100000:1.0:1789760710.534205:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534055424 00000100:00100000:1.0:1789760710.534267:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534055552 00000100:00100000:1.0:1789760710.534270:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534055552 00000100:00100000:1.0:1789760710.534314:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:1.0:1789760710.534324:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be418850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534055424:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760710.534345:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3839885312 avail 3821010944 00000100:00100000:1.0:1789760710.534388:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be418850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534055424:12345-0@lo:13:osp-pre-0-1.0 Request processed in 65us (334us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.534395:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:1.0:1789760710.534603:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 345 00000100:00100000:1.0:1789760710.534611:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534055552:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760710.534622:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3839885312 avail 3821010944 00000100:00100000:1.0:1789760710.534647:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534055552:12345-0@lo:13:osp-pre-0-0.0 Request processed in 37us (499us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.534662:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 345 00000100:00080000:1.0:1789760710.534684:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760710.535061:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b644700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534055552:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760710.535075:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b647480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534055424:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760710.537082:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0488421b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534055680:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760710.537150:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.543145:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd50700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3475:1876699534055808:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.543225:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.543267:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534056320:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.543333:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.543354:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3475:1876699534056832:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.543379:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.543402:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1789760710.543406:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3475:1876699534056192:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.543436:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.543452:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3475:1876699534056704:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.543471:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.543502:0:6460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534055808 00000100:00100000:1.0:1789760710.543508:0:6460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534055808 00000100:00100000:1.0:1789760710.543550:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 460 00000100:00100000:1.0:1789760710.543559:0:6460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04989030050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3475:x1876699534055808:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.543608:0:6460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04989030050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3475:x1876699534055808:12345-0@lo:400:kworker.0 Request processed in 49us (391us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.543615:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 460 00000100:00100000:1.0:1789760710.543646:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534056320 00000100:00100000:1.0:1789760710.543649:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534056320 00000100:00100000:1.0:1789760710.543857:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:1.0:1789760710.543863:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534056320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.543896:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534056320:12345-0@lo:400:kworker.0 Request processed in 33us (563us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.543903:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:1.0:1789760710.543918:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534056832 00000100:00100000:1.0:1789760710.543921:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534056832 00000100:00100000:1.0:1789760710.543950:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534056192 00000100:00100000:1.0:1789760710.543953:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534056192 00000100:00100000:1.0:1789760710.543966:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534056704 00000100:00100000:1.0:1789760710.543968:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534056704 00000100:00100000:1.0:1789760710.543983:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 833 00000100:00100000:1.0:1789760710.543989:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3475:x1876699534056832:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544014:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3475:x1876699534056832:12345-0@lo:400:kworker.0 Request processed in 26us (635us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.544020:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 833 00000100:00100000:1.0:1789760710.544029:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 834 00000100:00100000:1.0:1789760710.544034:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3475:x1876699534056192:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544054:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3475:x1876699534056192:12345-0@lo:400:kworker.0 Request processed in 20us (617us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.544060:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 834 00000100:00100000:1.0:1789760710.544068:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 835 00000100:00100000:1.0:1789760710.544073:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3475:x1876699534056704:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544091:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3475:x1876699534056704:12345-0@lo:400:kworker.0 Request processed in 18us (619us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.544096:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 835 00000100:00100000:1.0:1789760710.544130:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534056064:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544152:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.544168:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd50a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3473:1876699534056576:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544185:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.544202:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1789760710.544207:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd51f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3473:1876699534055936:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544239:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.544252:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd53100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3473:1876699534056448:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544269:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.544280:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd51500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534056960:0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544335:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.544359:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534056064 00000100:00100000:1.0:1789760710.544363:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534056064 00000100:00100000:1.0:1789760710.544382:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:1.0:1789760710.544387:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3473:x1876699534056064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544515:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3473:x1876699534056064:12345-0@lo:400:kworker.0 Request processed in 127us (362us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.544522:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00100000:1.0:1789760710.544538:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534056960 00000100:00100000:1.0:1789760710.544541:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534056960 00000100:00100000:1.0:1789760710.544565:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:1.0:1789760710.544571:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c32df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534056960:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544598:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c32df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534056960:12345-0@lo:400:kworker.0 Request processed in 27us (262us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.544603:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00100000:1.0:1789760710.544619:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534056576 00000100:00100000:1.0:1789760710.544621:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534056576 00000100:00100000:1.0:1789760710.544639:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534055936 00000100:00100000:1.0:1789760710.544641:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534055936 00000100:00100000:1.0:1789760710.544651:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534056448 00000100:00100000:1.0:1789760710.544793:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534056448 00000100:00100000:1.0:1789760710.544863:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 836 00000100:00100000:1.0:1789760710.544871:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3473:x1876699534056576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544902:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd53480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3473:x1876699534056576:12345-0@lo:400:kworker.0 Request processed in 33us (716us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.544909:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 836 00000100:00100000:1.0:1789760710.544919:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 837 00000100:00100000:1.0:1789760710.544925:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3473:x1876699534055936:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544951:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3473:x1876699534055936:12345-0@lo:400:kworker.0 Request processed in 26us (712us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.544956:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 837 00000100:00100000:1.0:1789760710.544966:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 838 00000100:00100000:1.0:1789760710.544972:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3473:x1876699534056448:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760710.544998:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3473:x1876699534056448:12345-0@lo:400:kworker.0 Request processed in 26us (727us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.545003:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 838 00000100:00100000:2.0:1789760710.549569:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760710.549591:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534057088:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760710.549668:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760710.554000:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534055680 00000100:00100000:1.0:1789760710.554013:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534055680 00000100:00100000:1.0:1789760710.554070:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534057088 00000100:00100000:1.0:1789760710.554073:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534057088 00000100:00100000:1.0:1789760710.554088:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 346 00000100:00100000:1.0:1789760710.554097:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534055680:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760710.554133:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760710.554213:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534055680:12345-0@lo:13:osp-pre-1-0.0 Request processed in 116us (17062us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.554316:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 346 00000100:00100000:1.0:1789760710.554333:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 347 00000100:00100000:1.0:1789760710.554339:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534057088:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760710.554351:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760710.554386:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534057088:12345-0@lo:13:osp-pre-1-1.0 Request processed in 48us (4718us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760710.554392:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 347 00000100:00100000:1.0:1789760710.554506:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0488421b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534055680:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760710.554554:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2dc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534057088:0@lo:13:osp-pre-1-1.0 00000100:00080000:2.0:1789760710.562129:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760710.562140:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760710.562143:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789760710.562313:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd51f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3473:1876699534055936:0@lo:400:kworker.0 00000100:00100000:2.0:1789760710.562334:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd53100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3473:1876699534056448:0@lo:400:kworker.0 00000100:00100000:2.0:1789760710.562344:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd51500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534056960:0@lo:400:kworker.0 00000100:00100000:2.0:1789760710.562355:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534056064:0@lo:400:kworker.0 00000100:00100000:2.0:1789760710.562365:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd50a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3473:1876699534056576:0@lo:400:kworker.0 00000100:00100000:2.0:1789760710.562414:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3475:1876699534056192:0@lo:400:kworker.0 00000100:00100000:2.0:1789760710.562426:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3475:1876699534056704:0@lo:400:kworker.0 00000100:00100000:2.0:1789760710.562435:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd50700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3475:1876699534055808:0@lo:400:kworker.0 00000100:00100000:2.0:1789760710.562445:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534056320:0@lo:400:kworker.0 00000100:00100000:2.0:1789760710.562457:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3475:1876699534056832:0@lo:400:kworker.0 00002000:00010000:2.0:1789760710.575901:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18427904 -> 18432000 00002000:00010000:2.0:1789760710.575905:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760710 -> 1789760711 00002000:00010000:2.0:1789760710.575908:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760710 -> 1789760711 00010000:00010000:2.0:1789760710.575921:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54214 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.575933:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54214 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.575942:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54214 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.575950:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760710.575954:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54214 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.575964:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54214 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.575985:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54214 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.575993:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54214 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18427904->18431999] (req 18427904->18431999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.576095:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d44c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508253312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 84130us (84401us total) trans 4294971819 rc 0/0 00000100:00100000:2.0:1789760710.576106:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4531 00000100:00100000:2.0:1789760710.576117:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689540670015 time 689596658585 state 0x2 ref 0 00000100:00100000:0.0:1789760710.584754:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760710.699543:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760710.699784:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508253568 00000100:00100000:2.0:1789760710.699793:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508253568 00000100:00100000:2.0:1789760710.699846:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689596919001 time 689689714596 state 0x0 ref 1 00000100:00100000:2.0:1789760710.699860:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689689975012 time 689689714596 state 0x1 ref 1 00000100:00100000:2.0:1789760710.699918:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689689975012 time 689804628674 state 0x0 ref 1 00000100:00100000:2.0:1789760710.699930:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c32f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508253568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760710.699984:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18436096, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9007 00010000:00010000:2.0:1789760710.699994:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54222 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.700012:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760710.700016:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.700023:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54222 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.700038:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54222 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760710.760618:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18436096 -> 18440192 00010000:00010000:2.0:1789760710.760637:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54222 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.761875:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54222 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.761893:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54222 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.761901:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760710.761907:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54222 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.761917:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54222 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.761939:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54222 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.761948:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54222 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18436096->18440191] (req 18436096->18440191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.762047:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c32f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508253568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 62112us (62510us total) trans 4294971821 rc 0/0 00000100:00100000:2.0:1789760710.762061:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4533 00000100:00100000:2.0:1789760710.762072:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689689975012 time 689804628674 state 0x2 ref 0 00000100:00100000:2.0:1789760710.767798:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760710.768561:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508253696 00000100:00100000:0.0:1789760710.768571:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508253696 00000100:00100000:0.0:1789760710.768627:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689689975012 time 689804628674 state 0x0 ref 1 00000100:00100000:0.0:1789760710.768643:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689804889090 time 689804628674 state 0x1 ref 1 00000100:00100000:0.0:1789760710.768853:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689804889090 time 689873562124 state 0x0 ref 1 00000100:00100000:0.0:1789760710.768868:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508253696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760710.768917:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18440192, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9007 00010000:00010000:0.0:1789760710.768926:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54229 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.768940:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:0.0:1789760710.768944:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760710.768951:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54229 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.768965:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54229 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760710.816099:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18440192 -> 18444288 00010000:00010000:0.0:1789760710.816114:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54229 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.816127:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54229 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.816142:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54229 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.816150:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760710.816156:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54229 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.816166:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54229 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.816186:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54229 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760710.816194:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54229 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18440192->18444287] (req 18440192->18444287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760710.816340:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508253696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47417us (48493us total) trans 4294971822 rc 0/0 00000100:00100000:0.0:1789760710.816350:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4534 00000100:00100000:0.0:1789760710.816359:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689804889090 time 689873562124 state 0x2 ref 0 00000100:00100000:2.0:1789760710.819994:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760710.872707:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760710.872857:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508253952 00000100:00100000:2.0:1789760710.872867:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508253952 00000100:00100000:2.0:1789760710.872925:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689873822540 time 689924914565 state 0x0 ref 1 00000100:00100000:2.0:1789760710.872942:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689925174981 time 689924914565 state 0x1 ref 1 00000100:00100000:2.0:1789760710.872974:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689925174981 time 689977684251 state 0x0 ref 1 00000100:00100000:2.0:1789760710.872986:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c32ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508253952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760710.873040:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18448384, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9007 00010000:00010000:2.0:1789760710.873049:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54237 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.873066:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=689s v=(0 0) 00010000:00010000:2.0:1789760710.873071:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.873078:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54237 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.873093:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54237 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760710.898730:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971824 is committed 00002000:00010000:2.0:1789760710.898911:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18448384 -> 18452480 00010000:00010000:2.0:1789760710.898925:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54237 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.898938:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54237 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.898948:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54237 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.898955:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760710.898960:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54237 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.898969:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54237 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.899081:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54237 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.899092:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54237 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18448384->18452479] (req 18448384->18452479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760710.899226:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c32ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508253952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26236us (26525us total) trans 4294971824 rc 0/0 00000100:00100000:2.0:1789760710.899240:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4536 00000100:00100000:2.0:1789760710.899250:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689925174981 time 689977684251 state 0x2 ref 0 00000100:00100000:0.0:1789760710.916980:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760710.917081:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508254080 00000100:00100000:2.0:1789760710.917091:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508254080 00000100:00100000:2.0:1789760710.917144:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689925174981 time 689977684251 state 0x0 ref 1 00000100:00100000:2.0:1789760710.917161:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689977944667 time 689977684251 state 0x1 ref 1 00000100:00100000:2.0:1789760710.917194:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689977944667 time 690021903932 state 0x0 ref 1 00000100:00100000:2.0:1789760710.917206:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508254080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760710.917255:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18452480, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9007 00010000:00010000:2.0:1789760710.917264:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5423e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.917279:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760710.917322:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760710.917329:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5423e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760710.917344:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5423e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760710.971619:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18452480 -> 18456576 00010000:00010000:1.0:1789760710.971638:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5423e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.971703:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5423e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.971712:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5423e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.971719:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760710.971724:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5423e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.971733:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5423e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.971757:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5423e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760710.971766:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5423e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18452480->18456575] (req 18452480->18456575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760710.971872:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508254080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54662us (54896us total) trans 4294971825 rc 0/0 00000100:00100000:1.0:1789760710.971884:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4537 00000100:00100000:1.0:1789760710.971893:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 689977944667 time 690021903932 state 0x2 ref 0 00000100:00100000:1.0:1789760710.975663:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760711.069339:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971826 is committed 00002000:00010000:1.0:1789760711.069512:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18456576 -> 18460672 00010000:00010000:1.0:1789760711.069526:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54245 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.069538:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54245 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.069548:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54245 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.069556:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760711.069561:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54245 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.069576:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54245 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.069596:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54245 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.069605:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54245 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18456576->18460671] (req 18456576->18460671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.069709:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508254208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 93838us (94051us total) trans 4294971826 rc 0/0 00000100:00100000:1.0:1789760711.069719:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4538 00000100:00100000:1.0:1789760711.069728:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690022164348 time 690080564984 state 0x2 ref 0 00000100:00100000:0.0:1789760711.081883:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760711.083142:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508254336 00000100:00100000:2.0:1789760711.083154:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508254336 00000100:00100000:2.0:1789760711.083249:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690022164348 time 690080564984 state 0x0 ref 1 00000100:00100000:2.0:1789760711.083265:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690080825400 time 690080564984 state 0x1 ref 1 00000100:00100000:2.0:1789760711.083347:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690080825400 time 690188057434 state 0x0 ref 1 00000100:00100000:2.0:1789760711.083360:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508254336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760711.083407:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18460672, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9007 00010000:00010000:2.0:1789760711.083416:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5424c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.083429:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00000100:00080000:1.0:1789760711.083433:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789760711.083433:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.083439:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5424c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.083452:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5424c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18460672->18464767] (req 18460672->18464767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.148761:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760711.153531:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508254464 00000100:00100000:1.0:1789760711.153547:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508254464 00000100:00100000:2.0:1789760711.156572:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690080825400 time 690188057434 state 0x0 ref 1 00000100:00100000:2.0:1789760711.156591:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690188317850 time 690188057434 state 0x1 ref 1 00000100:00100000:2.0:1789760711.156682:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690188317850 time 690261391890 state 0x0 ref 1 00000100:00100000:2.0:1789760711.156701:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508254464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760711.156752:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18464768, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9007 00010000:00010000:2.0:1789760711.156761:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54253 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.156776:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760711.156780:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.156786:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54253 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.156800:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54253 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760711.230671:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971828 is committed 00002000:00010000:2.0:1789760711.230835:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18464768 -> 18468864 00010000:00010000:2.0:1789760711.230850:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54253 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.230863:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54253 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.230874:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54253 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.230882:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760711.230887:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54253 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.230897:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54253 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.230918:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54253 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.230927:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54253 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18464768->18468863] (req 18464768->18468863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.231027:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508254464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 74324us (82270us total) trans 4294971828 rc 0/0 00000100:00100000:2.0:1789760711.231038:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4540 00000100:00100000:2.0:1789760711.231048:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690188317850 time 690261391890 state 0x2 ref 0 00000100:00100000:1.0:1789760711.234947:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760711.278892:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971829 is committed 00000100:00100000:1.0:1789760711.282124:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760711.374579:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760711.374683:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508254848 00000100:00100000:0.0:1789760711.374694:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508254848 00000100:00100000:0.0:1789760711.374749:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690340167031 time 690387077300 state 0x0 ref 1 00000100:00100000:0.0:1789760711.374763:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690387337716 time 690387077300 state 0x1 ref 1 00000100:00100000:0.0:1789760711.374875:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690387337716 time 690479584260 state 0x0 ref 1 00000100:00100000:0.0:1789760711.374890:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508254848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760711.374937:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18477056, mtime: 0x6aad94c7, atime: 0x0, ctime: 0x6aad94c7, blocks: 0x9007 00010000:00010000:0.0:1789760711.374947:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54268 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.374964:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760711.374969:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.374975:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54268 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.374990:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54268 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.422772:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760711.434476:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18477056 -> 18481152 00010000:00010000:0.0:1789760711.434495:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54268 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.434507:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54268 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.434516:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54268 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.434523:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760711.434528:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54268 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.434538:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54268 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.434557:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54268 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.434565:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54268 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18477056->18481151] (req 18477056->18481151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.434722:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508254848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59759us (60078us total) trans 4294971831 rc 0/0 00000100:00100000:0.0:1789760711.434735:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4543 00000100:00100000:0.0:1789760711.434746:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690387337716 time 690479584260 state 0x2 ref 0 00000001:00080000:0.0:1789760711.436501:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971831 is committed 00000100:00100000:1.0:1789760711.443845:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760711.494143:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971832 is committed 00002000:00010000:1.0:1789760711.494479:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18481152 -> 18485248 00010000:00010000:1.0:1789760711.494496:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5426f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.494515:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5426f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.494524:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5426f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.494532:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760711.494537:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5426f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.494546:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5426f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.494569:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5426f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.494578:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5426f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18481152->18485247] (req 18481152->18485247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.494857:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508254976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50799us (51015us total) trans 4294971832 rc 0/0 00000100:00100000:1.0:1789760711.494869:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4544 00000100:00100000:1.0:1789760711.494879:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690479844676 time 690548753012 state 0x2 ref 0 00000100:00100000:1.0:1789760711.504765:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760711.550886:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534057216:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789760711.550962:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760711.551547:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534057216 00000100:00100000:2.0:1789760711.551556:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534057216 00000100:00100000:2.0:1789760711.551601:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:2.0:1789760711.551610:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534057216:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789760711.551659:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789760711.551711:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534057216:12345-0@lo:41:osp-pre-0-1.0 Request processed in 102us (750us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760711.551719:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:1.0:1789760711.562478:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534057216:0@lo:41:osp-pre-0-1.0 00000001:00080000:2.0:1789760711.566920:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971833 is committed 00002000:00010000:0.0:1789760711.567105:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18485248 -> 18489344 00002000:00010000:0.0:1789760711.567110:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760711 -> 1789760712 00002000:00010000:0.0:1789760711.567112:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760711 -> 1789760712 00010000:00010000:0.0:1789760711.567125:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54276 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.567142:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54276 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.567152:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54276 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.567160:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760711.567166:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54276 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.567176:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54276 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.567197:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54276 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.567206:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54276 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18485248->18489343] (req 18485248->18489343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.567477:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508255104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 62442us (62716us total) trans 4294971833 rc 0/0 00000100:00100000:0.0:1789760711.567487:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4545 00000100:00100000:0.0:1789760711.567498:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690549013428 time 690609727237 state 0x2 ref 0 00000100:00100000:2.0:1789760711.572528:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760711.572614:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508255232 00000100:00100000:0.0:1789760711.572624:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508255232 00000100:00100000:0.0:1789760711.572680:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690549013428 time 690609727237 state 0x0 ref 1 00000100:00100000:0.0:1789760711.572695:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690609987653 time 690609727237 state 0x1 ref 1 00000100:00100000:0.0:1789760711.572730:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690609987653 time 690677440749 state 0x0 ref 1 00000100:00100000:0.0:1789760711.572742:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508255232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760711.572791:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18489344, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:0.0:1789760711.572800:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5427d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.572815:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760711.572819:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.572826:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5427d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.572841:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5427d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760711.672488:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971834 is committed 00002000:00010000:1.0:1789760711.672636:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18489344 -> 18493440 00010000:00010000:1.0:1789760711.672764:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5427d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.672777:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5427d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.672788:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5427d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.672796:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760711.672801:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5427d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.672810:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5427d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.672830:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5427d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.672838:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5427d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18489344->18493439] (req 18489344->18493439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.672973:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508255232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 100227us (100449us total) trans 4294971834 rc 0/0 00000100:00100000:1.0:1789760711.672984:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4546 00000100:00100000:1.0:1789760711.672994:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690609987653 time 690677440749 state 0x2 ref 0 00000100:00100000:1.0:1789760711.676872:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760711.713787:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18493440 -> 18497536 00010000:00010000:0.0:1789760711.713806:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54284 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.713819:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54284 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.713846:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54284 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.713854:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760711.713860:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54284 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.713869:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54284 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.713890:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54284 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.713899:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54284 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18493440->18497535] (req 18493440->18497535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760711.713995:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508255360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36871us (37127us total) trans 4294971835 rc 0/0 00000100:00100000:0.0:1789760711.714006:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4547 00000100:00100000:0.0:1789760711.714020:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690677701165 time 690781815669 state 0x2 ref 0 00000001:00080000:2.0:1789760711.714775:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971835 is committed 00000100:00100000:0.0:1789760711.721704:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760711.726807:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690677701165 time 690781815669 state 0x0 ref 1 00000100:00100000:0.0:1789760711.726825:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690782076085 time 690781815669 state 0x1 ref 1 00000100:00100000:0.0:1789760711.726920:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690782076085 time 690831629572 state 0x0 ref 1 00000100:00100000:0.0:1789760711.726937:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508255488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760711.726984:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18497536, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:0.0:1789760711.726994:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5428b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.727009:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:0.0:1789760711.727014:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.727021:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5428b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.727035:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5428b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760711.786211:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18497536 -> 18501632 00010000:00010000:2.0:1789760711.786229:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5428b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.786243:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5428b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.786252:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5428b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.786260:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760711.786265:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5428b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.786276:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5428b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.786446:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5428b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.786456:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5428b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18497536->18501631] (req 18497536->18501631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760711.786593:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508255488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59656us (64951us total) trans 4294971836 rc 0/0 00000100:00100000:2.0:1789760711.786608:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4548 00000100:00100000:2.0:1789760711.786618:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690782076085 time 690831629572 state 0x2 ref 0 00000100:00100000:1.0:1789760711.809679:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760711.809844:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508255616 00000100:00100000:2.0:1789760711.809853:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508255616 00000100:00100000:2.0:1789760711.809908:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690782076085 time 690831629572 state 0x0 ref 1 00000100:00100000:2.0:1789760711.809923:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690831889988 time 690831629572 state 0x1 ref 1 00000100:00100000:2.0:1789760711.809955:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690831889988 time 690914664702 state 0x0 ref 1 00000100:00100000:2.0:1789760711.809967:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508255616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760711.810018:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18501632, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:2.0:1789760711.810027:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54292 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.810042:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=690s v=(0 0) 00010000:00010000:2.0:1789760711.810047:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760711.810053:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54292 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760711.810069:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54292 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760711.921051:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971837 is committed 00002000:00010000:1.0:1789760711.921145:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18501632 -> 18505728 00010000:00010000:1.0:1789760711.921160:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54292 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.921172:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54292 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.921181:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54292 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.921188:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760711.921192:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54292 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.921201:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54292 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.921223:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54292 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760711.921229:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54292 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18501632->18505727] (req 18501632->18505727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760711.921954:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508255616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 111982us (112278us total) trans 4294971837 rc 0/0 00000100:00100000:1.0:1789760711.921965:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4549 00000100:00100000:1.0:1789760711.921975:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690831889988 time 690914664702 state 0x2 ref 0 00000100:00100000:2.0:1789760711.932275:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760711.932510:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508255744 00000100:00100000:0.0:1789760711.932520:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508255744 00000100:00100000:0.0:1789760711.932572:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690831889988 time 690914664702 state 0x0 ref 1 00000100:00100000:0.0:1789760711.932588:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690914925118 time 690914664702 state 0x1 ref 1 00000100:00100000:0.0:1789760711.932620:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690914925118 time 691037329905 state 0x0 ref 1 00000100:00100000:0.0:1789760711.932633:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508255744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760711.932694:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18505728, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:0.0:1789760711.932703:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54299 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.932720:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760711.932725:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760711.932731:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54299 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760711.932745:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54299 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.005845:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971838 is committed 00002000:00010000:0.0:1789760712.005939:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18505728 -> 18509824 00010000:00010000:0.0:1789760712.005955:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54299 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.005968:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54299 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.005978:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54299 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.005986:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760712.005991:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54299 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.006002:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54299 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.006026:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54299 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.006035:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54299 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18505728->18509823] (req 18505728->18509823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.006152:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508255744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 73515us (73882us total) trans 4294971838 rc 0/0 00000100:00100000:0.0:1789760712.006163:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4550 00000100:00100000:0.0:1789760712.006175:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690914925118 time 691037329905 state 0x2 ref 0 00000100:00100000:2.0:1789760712.010174:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760712.010341:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508255872 00000100:00100000:0.0:1789760712.010351:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508255872 00000100:00100000:0.0:1789760712.010472:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 690914925118 time 691037329905 state 0x0 ref 1 00000100:00100000:0.0:1789760712.010490:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691037590321 time 691037329905 state 0x1 ref 1 00000100:00100000:0.0:1789760712.010524:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691037590321 time 691115233568 state 0x0 ref 1 00000100:00100000:0.0:1789760712.010537:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508255872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760712.010589:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18509824, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:0.0:1789760712.010598:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe542a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.010613:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760712.010619:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.010625:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe542a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.010640:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe542a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760712.040704:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18509824 -> 18513920 00010000:00010000:2.0:1789760712.040724:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe542a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.040736:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe542a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.040744:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe542a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.040751:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760712.040755:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe542a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.040765:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe542a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.040784:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe542a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.040792:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe542a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18509824->18513919] (req 18509824->18513919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.040887:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508255872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30347us (30717us total) trans 4294971839 rc 0/0 00000100:00100000:2.0:1789760712.040897:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4551 00000100:00100000:2.0:1789760712.040906:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691037590321 time 691115233568 state 0x2 ref 0 00000100:00100000:2.0:1789760712.045056:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508256000 00000100:00100000:2.0:1789760712.045066:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508256000 00000100:00100000:2.0:1789760712.045118:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691037590321 time 691115233568 state 0x0 ref 1 00000100:00100000:2.0:1789760712.045132:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691115493984 time 691115233568 state 0x1 ref 1 00000100:00100000:2.0:1789760712.045167:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691115493984 time 691149877175 state 0x0 ref 1 00000100:00100000:2.0:1789760712.045180:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508256000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760712.045238:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18513920, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:2.0:1789760712.045248:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe542a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.045262:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760712.045267:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.045273:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe542a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.045327:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe542a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.047871:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971839 is committed 00000001:00080000:0.0:1789760712.076564:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971840 is committed 00002000:00010000:2.0:1789760712.076733:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18513920 -> 18518016 00010000:00010000:2.0:1789760712.076750:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe542a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.076762:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe542a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.076771:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe542a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.076779:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760712.076784:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe542a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.076793:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe542a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.076814:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe542a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.076822:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe542a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18513920->18518015] (req 18513920->18518015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.076956:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508256000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31773us (31981us total) trans 4294971840 rc 0/0 00000100:00100000:2.0:1789760712.076967:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4552 00000100:00100000:2.0:1789760712.076976:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691115493984 time 691149877175 state 0x2 ref 0 00000100:00100000:2.0:1789760712.080798:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760712.080895:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508256128 00000100:00100000:0.0:1789760712.080904:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508256128 00000100:00100000:0.0:1789760712.080955:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691115493984 time 691149877175 state 0x0 ref 1 00000100:00100000:0.0:1789760712.080971:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691150137591 time 691149877175 state 0x1 ref 1 00000100:00100000:0.0:1789760712.081027:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691150137591 time 691185736555 state 0x0 ref 1 00000100:00100000:0.0:1789760712.081039:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508256128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760712.081085:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18518016, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:0.0:1789760712.081094:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe542ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.081108:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:0.0:1789760712.081113:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760712.081119:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe542ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.081133:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe542ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18518016->18522111] (req 18518016->18522111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760712.108819:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971841 is committed 00000100:00100000:0.0:1789760712.113931:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760712.114009:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508256256 00000100:00100000:2.0:1789760712.114018:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508256256 00000100:00100000:2.0:1789760712.114070:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691150137591 time 691185736555 state 0x0 ref 1 00000100:00100000:2.0:1789760712.114085:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691185996971 time 691185736555 state 0x1 ref 1 00000100:00100000:2.0:1789760712.114115:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691185996971 time 691218825767 state 0x0 ref 1 00000100:00100000:2.0:1789760712.114128:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508256256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760712.114174:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18522112, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:2.0:1789760712.114184:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.114199:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760712.114213:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.114221:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.114234:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.147071:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971842 is committed 00002000:00010000:2.0:1789760712.147168:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18522112 -> 18526208 00010000:00010000:2.0:1789760712.147182:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.147194:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.147204:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.147211:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760712.147216:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.147225:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.147245:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.147253:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18522112->18526207] (req 18522112->18526207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.147532:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508256256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33400us (33604us total) trans 4294971842 rc 0/0 00000100:00100000:2.0:1789760712.147543:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4554 00000100:00100000:2.0:1789760712.147553:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691185996971 time 691218825767 state 0x2 ref 0 00000100:00100000:2.0:1789760712.150676:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760712.150804:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508256384 00000100:00100000:1.0:1789760712.150815:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508256384 00000100:00100000:1.0:1789760712.150866:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691185996971 time 691218825767 state 0x0 ref 1 00000100:00100000:1.0:1789760712.150882:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691219086183 time 691218825767 state 0x1 ref 1 00000100:00100000:1.0:1789760712.150914:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691219086183 time 691255624842 state 0x0 ref 1 00000100:00100000:1.0:1789760712.150927:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508256384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760712.150976:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18526208, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:1.0:1789760712.150986:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe542bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.151001:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760712.151005:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.151011:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe542bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.151031:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe542bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.196603:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971843 is committed 00002000:00010000:2.0:1789760712.196835:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18526208 -> 18530304 00010000:00010000:2.0:1789760712.196849:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe542bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.196861:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe542bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.196875:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe542bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.196882:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760712.196887:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe542bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.196895:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe542bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.196914:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe542bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.196921:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe542bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18526208->18530303] (req 18526208->18530303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.197012:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508256384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46082us (46342us total) trans 4294971843 rc 0/0 00000100:00100000:2.0:1789760712.197022:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4555 00000100:00100000:2.0:1789760712.197032:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691219086183 time 691255624842 state 0x2 ref 0 00000100:00100000:0.0:1789760712.200036:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:1.0:1789760712.241264:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18530304 -> 18534400 00010000:00010000:1.0:1789760712.241318:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe542c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.241333:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe542c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.241342:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe542c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.241350:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760712.241355:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe542c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.241364:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe542c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.241382:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe542c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.241390:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe542c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18530304->18534399] (req 18530304->18534399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.241635:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508256512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41216us (41602us total) trans 4294971844 rc 0/0 00000100:00100000:1.0:1789760712.241651:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4556 00000100:00100000:1.0:1789760712.241661:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691255885258 time 691305101492 state 0x2 ref 0 00000001:00080000:2.0:1789760712.242633:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971844 is committed 00000100:00100000:0.0:1789760712.245563:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760712.245871:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508256640 00000100:00100000:2.0:1789760712.245881:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508256640 00000100:00100000:2.0:1789760712.245939:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691255885258 time 691305101492 state 0x0 ref 1 00000100:00100000:2.0:1789760712.245957:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691305361908 time 691305101492 state 0x1 ref 1 00000100:00100000:2.0:1789760712.245990:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691305361908 time 691350700089 state 0x0 ref 1 00000100:00100000:2.0:1789760712.246002:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508256640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760712.246048:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18534400, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:2.0:1789760712.246057:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.246073:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760712.246077:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.246083:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.246098:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760712.277971:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971845 is committed 00002000:00010000:2.0:1789760712.278058:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18534400 -> 18538496 00010000:00010000:2.0:1789760712.278073:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.278086:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.278095:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.278109:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760712.278114:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.278122:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.278140:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.278148:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe542ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18534400->18538495] (req 18534400->18538495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.278371:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508256640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32364us (32812us total) trans 4294971845 rc 0/0 00000100:00100000:2.0:1789760712.278382:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4557 00000100:00100000:2.0:1789760712.278392:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691305361908 time 691350700089 state 0x2 ref 0 00000100:00100000:0.0:1789760712.282678:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760712.282750:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508256768 00000100:00100000:2.0:1789760712.282760:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508256768 00000100:00100000:2.0:1789760712.282812:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691305361908 time 691350700089 state 0x0 ref 1 00000100:00100000:2.0:1789760712.282827:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691350960505 time 691350700089 state 0x1 ref 1 00000100:00100000:2.0:1789760712.282862:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691350960505 time 691387572377 state 0x0 ref 1 00000100:00100000:2.0:1789760712.282878:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508256768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760712.282926:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18538496, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:2.0:1789760712.282935:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.282952:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760712.282956:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.282963:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.282977:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.316829:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971846 is committed 00002000:00010000:2.0:1789760712.316925:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18538496 -> 18542592 00010000:00010000:2.0:1789760712.316940:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.316954:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.316962:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.316970:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760712.316975:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.316985:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.317006:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.317014:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18538496->18542591] (req 18538496->18542591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760712.317146:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508256768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34265us (34472us total) trans 4294971846 rc 0/0 00000100:00100000:2.0:1789760712.317156:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4558 00000100:00100000:2.0:1789760712.317166:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691350960505 time 691387572377 state 0x2 ref 0 00000100:00100000:0.0:1789760712.320855:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760712.320932:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508256896 00000100:00100000:1.0:1789760712.320942:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508256896 00000100:00100000:1.0:1789760712.320992:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691350960505 time 691387572377 state 0x0 ref 1 00000100:00100000:1.0:1789760712.321007:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691387832793 time 691387572377 state 0x1 ref 1 00000100:00100000:1.0:1789760712.321035:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691387832793 time 691425745530 state 0x0 ref 1 00000100:00100000:1.0:1789760712.321047:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508256896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760712.321096:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18542592, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:1.0:1789760712.321104:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe542d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.321120:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760712.321125:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.321130:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe542d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.321144:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe542d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.357913:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971847 is committed 00002000:00010000:1.0:1789760712.357995:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18542592 -> 18546688 00010000:00010000:1.0:1789760712.358011:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe542d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.358023:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe542d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.358032:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe542d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.358039:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760712.358044:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe542d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.358054:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe542d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.358072:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe542d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.358079:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe542d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18542592->18546687] (req 18542592->18546687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.358194:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508256896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37143us (37342us total) trans 4294971847 rc 0/0 00000100:00100000:1.0:1789760712.358267:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4559 00000100:00100000:1.0:1789760712.358277:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691387832793 time 691425745530 state 0x2 ref 0 00000100:00100000:0.0:1789760712.362816:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760712.362918:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508257024 00000100:00100000:2.0:1789760712.362928:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508257024 00000100:00100000:2.0:1789760712.362983:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691387832793 time 691425745530 state 0x0 ref 1 00000100:00100000:2.0:1789760712.362998:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691426005946 time 691425745530 state 0x1 ref 1 00000100:00100000:2.0:1789760712.363061:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691426005946 time 691467770774 state 0x0 ref 1 00000100:00100000:2.0:1789760712.363074:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508257024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760712.363119:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18546688, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:2.0:1789760712.363127:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.363143:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760712.363148:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.363154:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.363169:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760712.397087:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18546688 -> 18550784 00010000:00010000:0.0:1789760712.397105:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.397119:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.397128:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.397136:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760712.397141:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.397150:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.397175:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760712.397183:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe542df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18546688->18550783] (req 18546688->18550783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760712.397318:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508257024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34207us (34473us total) trans 4294971848 rc 0/0 00000100:00100000:0.0:1789760712.397330:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4560 00000100:00100000:0.0:1789760712.397339:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691426005946 time 691467770774 state 0x2 ref 0 00000001:00080000:1.0:1789760712.397667:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971848 is committed 00000100:00100000:1.0:1789760712.401280:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760712.401573:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508257152 00000100:00100000:2.0:1789760712.401582:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508257152 00000100:00100000:2.0:1789760712.401636:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691426005946 time 691467770774 state 0x0 ref 1 00000100:00100000:2.0:1789760712.401717:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691468031190 time 691467770774 state 0x1 ref 1 00000100:00100000:2.0:1789760712.401773:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691468031190 time 691506483201 state 0x0 ref 1 00000100:00100000:2.0:1789760712.401786:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508257152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760712.401833:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18550784, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:2.0:1789760712.401842:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe542e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.401858:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760712.401863:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.401870:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe542e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.401890:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe542e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760712.430154:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18550784 -> 18554880 00010000:00010000:1.0:1789760712.430173:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe542e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.430186:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe542e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.430195:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe542e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.430211:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760712.430216:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe542e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.430226:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe542e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.430245:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe542e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.430252:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe542e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18550784->18554879] (req 18550784->18554879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.430381:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508257152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28592us (29106us total) trans 4294971849 rc 0/0 00000100:00100000:1.0:1789760712.430392:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4561 00000100:00100000:1.0:1789760712.430412:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691468031190 time 691506483201 state 0x2 ref 0 00000100:00100000:0.0:1789760712.436529:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760712.466813:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971850 is committed 00002000:00010000:1.0:1789760712.466901:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18554880 -> 18558976 00010000:00010000:1.0:1789760712.466915:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe542ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.466929:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe542ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.466938:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe542ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.466945:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760712.466950:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe542ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.466960:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe542ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.466978:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe542ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.466991:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe542ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18554880->18558975] (req 18554880->18558975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.467106:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508257280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29972us (30580us total) trans 4294971850 rc 0/0 00000100:00100000:1.0:1789760712.467115:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4562 00000100:00100000:1.0:1789760712.467125:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691506743617 time 691541829216 state 0x2 ref 0 00000100:00100000:0.0:1789760712.472391:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760712.472535:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508257408 00000100:00100000:2.0:1789760712.472545:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508257408 00000100:00100000:2.0:1789760712.472599:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691506743617 time 691541829216 state 0x0 ref 1 00000100:00100000:2.0:1789760712.472614:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691542089632 time 691541829216 state 0x1 ref 1 00000100:00100000:2.0:1789760712.472688:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691542089632 time 691577397151 state 0x0 ref 1 00000100:00100000:2.0:1789760712.472700:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508257408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760712.472747:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18558976, mtime: 0x6aad94c8, atime: 0x0, ctime: 0x6aad94c8, blocks: 0x9007 00010000:00010000:2.0:1789760712.472757:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe542f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.472772:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760712.472777:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.472782:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe542f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18558976->18563071] (req 18558976->18563071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.472796:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe542f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18558976->18563071] (req 18558976->18563071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.502788:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508257536 00000100:00100000:1.0:1789760712.502797:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508257536 00000100:00100000:1.0:1789760712.502844:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691542089632 time 691577397151 state 0x0 ref 1 00000100:00100000:1.0:1789760712.502858:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691577657567 time 691577397151 state 0x1 ref 1 00000100:00100000:1.0:1789760712.502888:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691577657567 time 691607598868 state 0x0 ref 1 00000100:00100000:1.0:1789760712.502901:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508257536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760712.502945:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18563072, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:1.0:1789760712.502954:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe542fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.502969:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760712.502974:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.502980:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe542fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18563072->18567167] (req 18563072->18567167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.503000:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe542fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18563072->18567167] (req 18563072->18567167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.548646:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971852 is committed 00000100:00100000:2.0:1789760712.553589:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508257664 00000100:00100000:2.0:1789760712.553599:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508257664 00000100:00100000:2.0:1789760712.553717:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691577657567 time 691607598868 state 0x0 ref 1 00000100:00100000:2.0:1789760712.553736:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691607859284 time 691607598868 state 0x1 ref 1 00000100:00100000:2.0:1789760712.553773:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691607859284 time 691658479276 state 0x0 ref 1 00000100:00100000:2.0:1789760712.553785:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508257664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760712.553834:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18567168, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:2.0:1789760712.553842:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54302 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.553858:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:2.0:1789760712.553862:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760712.553868:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54302 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760712.553882:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54302 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760712.601126:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18567168 -> 18571264 00010000:00010000:1.0:1789760712.601141:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54302 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.601154:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54302 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.601163:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54302 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.601171:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760712.601176:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54302 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.601185:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54302 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.601204:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54302 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.601224:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54302 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18567168->18571263] (req 18567168->18571263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.601388:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508257664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47599us (47876us total) trans 4294971853 rc 0/0 00000100:00100000:1.0:1789760712.601398:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4565 00000100:00100000:1.0:1789760712.601408:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691607859284 time 691658479276 state 0x2 ref 0 00000100:00100000:0.0:1789760712.607522:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760712.641730:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971854 is committed 00002000:00010000:1.0:1789760712.642195:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18571264 -> 18575360 00010000:00010000:1.0:1789760712.642211:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54309 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.642231:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54309 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.642241:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54309 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.642248:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760712.642252:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54309 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.642262:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54309 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.642282:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54309 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.642521:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54309 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18571264->18575359] (req 18571264->18575359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.642664:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508257792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34797us (35148us total) trans 4294971854 rc 0/0 00000100:00100000:1.0:1789760712.642674:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4566 00000100:00100000:1.0:1789760712.642684:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691658739692 time 691712561665 state 0x2 ref 0 00000100:00100000:1.0:1789760712.648350:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508257920 00000100:00100000:1.0:1789760712.648363:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508257920 00000100:00100000:1.0:1789760712.648440:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691658739692 time 691712561665 state 0x0 ref 1 00000100:00100000:1.0:1789760712.648458:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691712822081 time 691712561665 state 0x1 ref 1 00000100:00100000:1.0:1789760712.648493:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691712822081 time 691753203403 state 0x0 ref 1 00000100:00100000:1.0:1789760712.648506:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508257920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760712.648557:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18575360, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:1.0:1789760712.648567:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54310 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.648582:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760712.648587:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.648593:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54310 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.648612:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54310 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18575360->18579455] (req 18575360->18579455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.679867:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971855 is committed 00000100:00100000:1.0:1789760712.695082:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508258048 00000100:00100000:1.0:1789760712.695092:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508258048 00000100:00100000:1.0:1789760712.695144:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691712822081 time 691753203403 state 0x0 ref 1 00000100:00100000:1.0:1789760712.695157:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691753463819 time 691753203403 state 0x1 ref 1 00000100:00100000:1.0:1789760712.695182:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691753463819 time 691799894185 state 0x0 ref 1 00000100:00100000:1.0:1789760712.695191:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508258048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760712.695238:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18579456, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:1.0:1789760712.695245:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54317 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.695258:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760712.695261:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.695266:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54317 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.695276:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54317 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18579456->18583551] (req 18579456->18583551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760712.728550:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971856 is committed 00000100:00100000:0.0:1789760712.732058:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760712.732373:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508258176 00000100:00100000:1.0:1789760712.732384:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508258176 00000100:00100000:1.0:1789760712.732439:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691753463819 time 691799894185 state 0x0 ref 1 00000100:00100000:1.0:1789760712.732459:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691800154601 time 691799894185 state 0x1 ref 1 00000100:00100000:1.0:1789760712.732490:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691800154601 time 691837200429 state 0x0 ref 1 00000100:00100000:1.0:1789760712.732504:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508258176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760712.732558:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18583552, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:1.0:1789760712.732568:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5431e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.732584:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789760712.732588:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.732594:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5431e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.732609:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5431e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760712.815381:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18583552 -> 18587648 00010000:00010000:1.0:1789760712.815403:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5431e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.815417:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5431e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.815431:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5431e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.815438:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760712.815443:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5431e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.815451:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5431e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.815472:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5431e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.815480:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5431e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18583552->18587647] (req 18583552->18587647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.815581:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508258176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 83074us (83527us total) trans 4294971857 rc 0/0 00000100:00100000:1.0:1789760712.815593:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4569 00000100:00100000:1.0:1789760712.815604:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691800154601 time 691837200429 state 0x2 ref 0 00000001:00080000:0.0:1789760712.816399:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971857 is committed 00000100:00100000:1.0:1789760712.825933:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760712.940245:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971858 is committed 00002000:00010000:1.0:1789760712.942766:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18587648 -> 18591744 00010000:00010000:1.0:1789760712.942783:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54325 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.942800:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54325 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.942809:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54325 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.942816:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760712.942821:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54325 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.942830:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54325 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.942850:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54325 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.942857:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54325 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18587648->18591743] (req 18587648->18591743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760712.942963:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508258304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 114049us (117035us total) trans 4294971858 rc 0/0 00000100:00100000:1.0:1789760712.942973:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4570 00000100:00100000:1.0:1789760712.942982:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691837460845 time 691933609279 state 0x2 ref 0 00000100:00100000:1.0:1789760712.955527:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508258432 00000100:00100000:1.0:1789760712.955544:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508258432 00000100:00100000:1.0:1789760712.955605:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691837460845 time 691933609279 state 0x0 ref 1 00000100:00100000:1.0:1789760712.955619:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691933869695 time 691933609279 state 0x1 ref 1 00000100:00100000:1.0:1789760712.955640:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691933869695 time 692060350855 state 0x0 ref 1 00000100:00100000:1.0:1789760712.955652:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508258432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760712.955698:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18591744, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:1.0:1789760712.955706:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5432c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.955722:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760712.955726:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760712.955732:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5432c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760712.955745:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5432c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.056066:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971859 is committed 00002000:00010000:1.0:1789760713.057195:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18591744 -> 18595840 00010000:00010000:1.0:1789760713.057226:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5432c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.057237:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5432c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.057251:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5432c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.057258:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760713.057262:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5432c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.057272:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5432c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.057333:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5432c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.057342:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5432c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18591744->18595839] (req 18591744->18595839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.057434:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508258432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 101779us (102028us total) trans 4294971859 rc 0/0 00000100:00100000:1.0:1789760713.057444:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4571 00000100:00100000:1.0:1789760713.057453:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691933869695 time 692060350855 state 0x2 ref 0 00000100:00100000:2.0:1789760713.063017:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.063151:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508258560 00000100:00100000:1.0:1789760713.063161:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508258560 00000100:00100000:1.0:1789760713.063240:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 691933869695 time 692060350855 state 0x0 ref 1 00000100:00100000:1.0:1789760713.063255:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692060611271 time 692060350855 state 0x1 ref 1 00000100:00100000:1.0:1789760713.063329:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692060611271 time 692167994547 state 0x0 ref 1 00000100:00100000:1.0:1789760713.063343:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508258560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.063387:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18595840, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:1.0:1789760713.063395:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54333 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.063410:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.063414:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.063420:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54333 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.063433:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54333 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760713.064008:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760713.111837:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18595840 -> 18599936 00010000:00010000:1.0:1789760713.111858:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54333 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.111869:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54333 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.111879:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54333 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.111886:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760713.111891:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54333 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.111900:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54333 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.111918:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54333 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.111925:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54333 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18595840->18599935] (req 18595840->18599935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.112015:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508258560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48671us (49003us total) trans 4294971860 rc 0/0 00000100:00100000:1.0:1789760713.112025:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4572 00000100:00100000:1.0:1789760713.112035:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692060611271 time 692167994547 state 0x2 ref 0 00000100:00100000:2.0:1789760713.117678:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.117763:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508258688 00000100:00100000:1.0:1789760713.117773:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508258688 00000100:00100000:1.0:1789760713.117825:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692060611271 time 692167994547 state 0x0 ref 1 00000100:00100000:1.0:1789760713.117840:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692168254963 time 692167994547 state 0x1 ref 1 00000100:00100000:1.0:1789760713.117870:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692168254963 time 692222580502 state 0x0 ref 1 00000100:00100000:1.0:1789760713.117882:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508258688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.117927:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18599936, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:1.0:1789760713.117935:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5433a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.117949:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.117953:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.117960:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5433a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.117974:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5433a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760713.162065:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18599936 -> 18604032 00010000:00010000:2.0:1789760713.162086:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5433a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.162100:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5433a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.162108:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5433a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.162321:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760713.162326:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5433a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.162336:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5433a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.162356:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5433a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.162364:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5433a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18599936->18604031] (req 18599936->18604031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.162452:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508258688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44565us (44776us total) trans 4294971861 rc 0/0 00000100:00100000:2.0:1789760713.162464:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4573 00000100:00100000:2.0:1789760713.162475:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692168254963 time 692222580502 state 0x2 ref 0 00000001:00080000:2.0:1789760713.164550:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971861 is committed 00000100:00100000:2.0:1789760713.170229:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.170391:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508258816 00000100:00100000:1.0:1789760713.170401:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508258816 00000100:00100000:1.0:1789760713.170455:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692168254963 time 692222580502 state 0x0 ref 1 00000100:00100000:1.0:1789760713.170470:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692222840918 time 692222580502 state 0x1 ref 1 00000100:00100000:1.0:1789760713.170498:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692222840918 time 692275208468 state 0x0 ref 1 00000100:00100000:1.0:1789760713.170509:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508258816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.170560:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18604032, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:1.0:1789760713.170569:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54341 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.170584:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.170589:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.170595:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54341 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.170608:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54341 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.218428:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971862 is committed 00002000:00010000:1.0:1789760713.218538:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18604032 -> 18608128 00010000:00010000:1.0:1789760713.218553:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54341 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.218565:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54341 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.218574:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54341 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.218582:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760713.218586:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54341 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.218596:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54341 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.218614:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54341 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.218622:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54341 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18604032->18608127] (req 18604032->18608127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.218836:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508258816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48324us (48612us total) trans 4294971862 rc 0/0 00000100:00100000:1.0:1789760713.218846:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4574 00000100:00100000:1.0:1789760713.218855:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692222840918 time 692275208468 state 0x2 ref 0 00000100:00100000:0.0:1789760713.221985:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.222073:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508258944 00000100:00100000:1.0:1789760713.222083:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508258944 00000100:00100000:1.0:1789760713.222149:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692222840918 time 692275208468 state 0x0 ref 1 00000100:00100000:1.0:1789760713.222165:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692275468884 time 692275208468 state 0x1 ref 1 00000100:00100000:1.0:1789760713.222194:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692275468884 time 692326904481 state 0x0 ref 1 00000100:00100000:1.0:1789760713.222207:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508258944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.222254:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18608128, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:1.0:1789760713.222263:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54348 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.222279:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.222324:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.222332:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54348 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18608128->18612223] (req 18608128->18612223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.222346:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54348 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18608128->18612223] (req 18608128->18612223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.269275:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971863 is committed 00000100:00100000:2.0:1789760713.286223:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.286380:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508259072 00000100:00100000:1.0:1789760713.286391:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508259072 00000100:00100000:1.0:1789760713.286446:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692275468884 time 692326904481 state 0x0 ref 1 00000100:00100000:1.0:1789760713.286462:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692327164897 time 692326904481 state 0x1 ref 1 00000100:00100000:1.0:1789760713.286538:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692327164897 time 692391247662 state 0x0 ref 1 00000100:00100000:1.0:1789760713.286552:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508259072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.286820:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18612224, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:1.0:1789760713.286829:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5434f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.286845:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.286850:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.286858:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5434f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.286873:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5434f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.335510:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971864 is committed 00002000:00010000:2.0:1789760713.335853:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18612224 -> 18616320 00010000:00010000:2.0:1789760713.335869:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5434f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.335882:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5434f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.336008:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5434f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.336015:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760713.336020:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5434f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.336028:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5434f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.336047:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5434f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.336054:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5434f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18612224->18616319] (req 18612224->18616319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.336256:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508259072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49702us (50038us total) trans 4294971864 rc 0/0 00000100:00100000:2.0:1789760713.336266:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4576 00000100:00100000:2.0:1789760713.336276:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692327164897 time 692391247662 state 0x2 ref 0 00000100:00100000:2.0:1789760713.342735:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.342805:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508259200 00000100:00100000:1.0:1789760713.342815:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508259200 00000100:00100000:1.0:1789760713.342869:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692327164897 time 692391247662 state 0x0 ref 1 00000100:00100000:1.0:1789760713.342897:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692391508078 time 692391247662 state 0x1 ref 1 00000100:00100000:1.0:1789760713.343035:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692391508078 time 692447745128 state 0x0 ref 1 00000100:00100000:1.0:1789760713.343049:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508259200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.343104:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18616320, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:1.0:1789760713.343189:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54356 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.343207:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.343212:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.343220:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54356 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.343234:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54356 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.403604:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971865 is committed 00002000:00010000:0.0:1789760713.404119:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18616320 -> 18620416 00010000:00010000:0.0:1789760713.404135:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54356 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.404155:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54356 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.404164:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54356 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.404171:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760713.404176:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54356 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.404184:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54356 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.404203:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54356 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.404211:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54356 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18616320->18620415] (req 18616320->18620415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.404438:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508259200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61386us (61709us total) trans 4294971865 rc 0/0 00000100:00100000:0.0:1789760713.404448:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4577 00000100:00100000:0.0:1789760713.404458:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692391508078 time 692447745128 state 0x2 ref 0 00000100:00100000:0.0:1789760713.415125:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00010000:00010000:0.0:1789760713.474115:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760713.476418:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971866 is committed 00002000:00010000:1.0:1789760713.476586:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18620416 -> 18624512 00010000:00010000:1.0:1789760713.476601:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5435d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.476614:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5435d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.476623:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5435d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.476640:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760713.476646:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5435d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.476654:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5435d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.476674:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5435d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.476682:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5435d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18620416->18624511] (req 18620416->18624511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.476781:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508259328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61402us (61661us total) trans 4294971866 rc 0/0 00000100:00100000:1.0:1789760713.476791:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4578 00000100:00100000:1.0:1789760713.476801:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692448005544 time 692520070446 state 0x2 ref 0 00000100:00100000:2.0:1789760713.480478:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.480560:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508259456 00000100:00100000:1.0:1789760713.480571:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508259456 00000100:00100000:1.0:1789760713.480627:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692448005544 time 692520070446 state 0x0 ref 1 00000100:00100000:1.0:1789760713.480652:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692520330862 time 692520070446 state 0x1 ref 1 00000100:00100000:1.0:1789760713.480681:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692520330862 time 692585391827 state 0x0 ref 1 00000100:00100000:1.0:1789760713.480693:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508259456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.480750:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18624512, mtime: 0x6aad94c9, atime: 0x0, ctime: 0x6aad94c9, blocks: 0x9007 00010000:00010000:1.0:1789760713.480764:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54364 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.480781:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.480785:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.480791:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54364 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.480806:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54364 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760713.508496:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18624512 -> 18628608 00002000:00010000:2.0:1789760713.508505:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760713 -> 1789760714 00002000:00010000:2.0:1789760713.508508:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760713 -> 1789760714 00010000:00010000:2.0:1789760713.508520:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54364 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.508533:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54364 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.508542:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54364 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.508550:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760713.508555:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54364 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.508572:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54364 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.508592:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54364 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.508599:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54364 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18624512->18628607] (req 18624512->18628607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.508693:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508259456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27997us (28220us total) trans 4294971867 rc 0/0 00000100:00100000:2.0:1789760713.508703:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4579 00000100:00100000:2.0:1789760713.508713:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692520330862 time 692585391827 state 0x2 ref 0 00000100:00100000:2.0:1789760713.526506:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.526667:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508259584 00000100:00100000:1.0:1789760713.526678:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508259584 00000100:00100000:1.0:1789760713.526734:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692520330862 time 692585391827 state 0x0 ref 1 00000100:00100000:1.0:1789760713.526750:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692585652243 time 692585391827 state 0x1 ref 1 00000100:00100000:1.0:1789760713.526960:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692585652243 time 692631668477 state 0x0 ref 1 00000100:00100000:1.0:1789760713.526976:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508259584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.527023:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18628608, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:1.0:1789760713.527031:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5436b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.527045:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.527048:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.527054:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5436b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.527081:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5436b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.567463:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971868 is committed 00002000:00010000:2.0:1789760713.567698:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18628608 -> 18632704 00010000:00010000:2.0:1789760713.567713:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5436b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.567726:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5436b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.567734:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5436b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.567746:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760713.567752:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5436b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.567761:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5436b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.567988:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5436b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.567997:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5436b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18628608->18632703] (req 18628608->18632703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.568098:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508259584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41116us (41593us total) trans 4294971868 rc 0/0 00000100:00100000:2.0:1789760713.568109:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4580 00000100:00100000:2.0:1789760713.568119:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692585652243 time 692631668477 state 0x2 ref 0 00000100:00100000:0.0:1789760713.577355:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.577390:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508259712 00000100:00100000:1.0:1789760713.577401:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508259712 00000100:00100000:1.0:1789760713.577459:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692585652243 time 692631668477 state 0x0 ref 1 00000100:00100000:1.0:1789760713.577475:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692631928893 time 692631668477 state 0x1 ref 1 00000100:00100000:1.0:1789760713.577503:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692631928893 time 692682213598 state 0x0 ref 1 00000100:00100000:1.0:1789760713.577527:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508259712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.577576:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18632704, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:1.0:1789760713.577585:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54372 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.577601:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.577605:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.577612:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54372 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.577626:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54372 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.599593:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534057344:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760713.599712:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760713.599903:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534057344 00000100:00100000:1.0:1789760713.599911:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534057344 00000100:00100000:1.0:1789760713.599990:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:1.0:1789760713.599999:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534057344:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760713.600033:0:7358:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760713.600083:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534057344:12345-0@lo:41:osp-pre-1-0.0 Request processed in 85us (371us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760713.600091:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1789760713.600152:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2dd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534057344:0@lo:41:osp-pre-1-0.0 00002000:00010000:1.0:1789760713.600717:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18632704 -> 18636800 00010000:00010000:1.0:1789760713.600730:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54372 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.600743:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54372 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.600752:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54372 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.600760:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760713.600765:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54372 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.600777:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54372 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.600851:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54372 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.600866:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54372 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18632704->18636799] (req 18632704->18636799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.600960:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508259712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23444us (23612us total) trans 4294971869 rc 0/0 00000100:00100000:1.0:1789760713.600969:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4581 00000100:00100000:1.0:1789760713.600977:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692631928893 time 692682213598 state 0x2 ref 0 00000100:00100000:1.0:1789760713.603963:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760713.604057:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508259840 00000100:00100000:2.0:1789760713.604065:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508259840 00000100:00100000:2.0:1789760713.604114:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692631928893 time 692682213598 state 0x0 ref 1 00000100:00100000:2.0:1789760713.604132:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692682474014 time 692682213598 state 0x1 ref 1 00000100:00100000:2.0:1789760713.604189:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692682474014 time 692708899422 state 0x0 ref 1 00000100:00100000:2.0:1789760713.604201:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508259840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760713.604245:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18636800, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:2.0:1789760713.604254:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54379 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.604269:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1789760713.604273:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.604279:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54379 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.604339:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54379 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.625560:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971870 is committed 00002000:00010000:0.0:1789760713.625732:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18636800 -> 18640896 00010000:00010000:0.0:1789760713.625748:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54379 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.625760:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54379 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.625770:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54379 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.625777:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760713.625781:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54379 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.625790:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54379 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.625811:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54379 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.625819:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54379 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18636800->18640895] (req 18636800->18640895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.625947:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508259840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 21743us (21988us total) trans 4294971870 rc 0/0 00000100:00100000:0.0:1789760713.625962:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4582 00000100:00100000:0.0:1789760713.625972:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692682474014 time 692708899422 state 0x2 ref 0 00000100:00100000:2.0:1789760713.630475:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.630526:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508259968 00000100:00100000:1.0:1789760713.630536:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508259968 00000100:00100000:1.0:1789760713.630590:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692682474014 time 692708899422 state 0x0 ref 1 00000100:00100000:1.0:1789760713.630605:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692709159838 time 692708899422 state 0x1 ref 1 00000100:00100000:1.0:1789760713.630633:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692709159838 time 692735343793 state 0x0 ref 1 00000100:00100000:1.0:1789760713.630646:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508259968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.630697:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18640896, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:1.0:1789760713.630707:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54380 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.630721:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.630726:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.630733:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54380 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.630749:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54380 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760713.647342:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18640896 -> 18644992 00010000:00010000:1.0:1789760713.647359:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54380 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.647373:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54380 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.647382:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54380 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.647389:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760713.647394:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54380 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.647403:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54380 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.647423:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54380 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.647431:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54380 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18640896->18644991] (req 18640896->18644991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.647610:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508259968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16960us (17135us total) trans 4294971871 rc 0/0 00000100:00100000:1.0:1789760713.647620:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4583 00000100:00100000:1.0:1789760713.647631:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692709159838 time 692735343793 state 0x2 ref 0 00000100:00100000:1.0:1789760713.651915:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760713.652011:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508260096 00000100:00100000:2.0:1789760713.652022:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508260096 00000100:00100000:2.0:1789760713.652068:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692709159838 time 692735343793 state 0x0 ref 1 00000100:00100000:2.0:1789760713.652079:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692735604209 time 692735343793 state 0x1 ref 1 00000100:00100000:2.0:1789760713.652130:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692735604209 time 692756841175 state 0x0 ref 1 00000100:00100000:2.0:1789760713.652141:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508260096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760713.652180:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18644992, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:2.0:1789760713.652186:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54387 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.652199:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:2.0:1789760713.652202:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760713.652206:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54387 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.652220:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54387 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760713.682066:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18644992 -> 18649088 00010000:00010000:0.0:1789760713.682081:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54387 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.682095:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54387 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.682104:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54387 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.682112:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760713.682117:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54387 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.682127:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54387 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.682147:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54387 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.682154:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54387 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18644992->18649087] (req 18644992->18649087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.682334:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508260096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30188us (30424us total) trans 4294971872 rc 0/0 00000100:00100000:0.0:1789760713.682345:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4584 00000100:00100000:0.0:1789760713.682355:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692735604209 time 692756841175 state 0x2 ref 0 00000100:00100000:2.0:1789760713.686410:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760713.686844:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508260224 00000100:00100000:0.0:1789760713.686855:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508260224 00000100:00100000:0.0:1789760713.686912:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692735604209 time 692756841175 state 0x0 ref 1 00000100:00100000:0.0:1789760713.686927:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692757101591 time 692756841175 state 0x1 ref 1 00000100:00100000:0.0:1789760713.686961:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692757101591 time 692791671289 state 0x0 ref 1 00000100:00100000:0.0:1789760713.686973:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508260224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760713.687022:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18649088, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:0.0:1789760713.687030:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5438e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.687046:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760713.687050:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.687056:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5438e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.687069:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5438e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.703411:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971873 is committed 00002000:00010000:1.0:1789760713.703553:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18649088 -> 18653184 00010000:00010000:1.0:1789760713.703570:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5438e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.703584:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5438e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.703595:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5438e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.703603:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760713.703608:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5438e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.703618:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5438e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.703653:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5438e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.703668:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5438e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18649088->18653183] (req 18649088->18653183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.703761:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508260224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16784us (17355us total) trans 4294971873 rc 0/0 00000100:00100000:1.0:1789760713.703772:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4585 00000100:00100000:1.0:1789760713.703782:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692757101591 time 692791671289 state 0x2 ref 0 00000100:00100000:2.0:1789760713.710781:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.710898:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508260352 00000100:00100000:1.0:1789760713.710909:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508260352 00000100:00100000:1.0:1789760713.710970:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692757101591 time 692791671289 state 0x0 ref 1 00000100:00100000:1.0:1789760713.710986:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692791931705 time 692791671289 state 0x1 ref 1 00000100:00100000:1.0:1789760713.711013:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692791931705 time 692815723732 state 0x0 ref 1 00000100:00100000:1.0:1789760713.711028:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508260352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.711076:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18653184, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:1.0:1789760713.711087:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54395 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.711144:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.711150:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.711159:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54395 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.711185:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54395 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.742198:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971874 is committed 00002000:00010000:0.0:1789760713.742418:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18653184 -> 18657280 00010000:00010000:0.0:1789760713.742433:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54395 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.742447:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54395 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.742455:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54395 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.742463:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760713.742469:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54395 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.742478:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54395 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.742496:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54395 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.742503:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54395 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18653184->18657279] (req 18653184->18657279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.742601:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508260352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31572us (31824us total) trans 4294971874 rc 0/0 00000100:00100000:0.0:1789760713.742612:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4586 00000100:00100000:0.0:1789760713.742622:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692791931705 time 692815723732 state 0x2 ref 0 00000100:00100000:2.0:1789760713.748904:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760713.749706:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508260480 00000100:00100000:0.0:1789760713.749717:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508260480 00000100:00100000:0.0:1789760713.749796:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692791931705 time 692815723732 state 0x0 ref 1 00000100:00100000:0.0:1789760713.749812:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692815984148 time 692815723732 state 0x1 ref 1 00000100:00100000:0.0:1789760713.749843:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692815984148 time 692854553382 state 0x0 ref 1 00000100:00100000:0.0:1789760713.749855:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508260480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760713.749906:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18657280, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:0.0:1789760713.749915:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5439c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.749930:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760713.749935:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.749941:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5439c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.749954:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5439c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760713.782881:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18657280 -> 18661376 00010000:00010000:0.0:1789760713.782906:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5439c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.782921:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5439c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.782929:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5439c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.782936:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760713.782940:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5439c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.782949:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5439c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.782969:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5439c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.782976:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5439c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18657280->18661375] (req 18657280->18661375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.783070:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508260480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33211us (34172us total) trans 4294971875 rc 0/0 00000100:00100000:0.0:1789760713.783079:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4587 00000100:00100000:0.0:1789760713.783089:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692815984148 time 692854553382 state 0x2 ref 0 00000100:00100000:2.0:1789760713.793405:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.793522:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508260608 00000100:00100000:1.0:1789760713.793532:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508260608 00000100:00100000:1.0:1789760713.793584:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692815984148 time 692854553382 state 0x0 ref 1 00000100:00100000:1.0:1789760713.793599:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692854813798 time 692854553382 state 0x1 ref 1 00000100:00100000:1.0:1789760713.793630:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692854813798 time 692898340544 state 0x0 ref 1 00000100:00100000:1.0:1789760713.793642:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d46850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508260608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.793689:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18661376, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:1.0:1789760713.793698:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe543a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.793719:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:1.0:1789760713.793725:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.793731:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe543a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.793746:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe543a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760713.826572:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18661376 -> 18665472 00010000:00010000:2.0:1789760713.826592:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe543a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.826606:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe543a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.826615:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe543a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.826690:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760713.826695:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe543a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.826705:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe543a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.826724:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe543a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760713.826733:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe543a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18661376->18665471] (req 18661376->18665471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760713.826867:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d46850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508260608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33222us (33464us total) trans 4294971876 rc 0/0 00000100:00100000:2.0:1789760713.826878:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4588 00000100:00100000:2.0:1789760713.826888:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692854813798 time 692898340544 state 0x2 ref 0 00000100:00100000:0.0:1789760713.830423:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508260736 00000100:00100000:0.0:1789760713.830432:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508260736 00000100:00100000:0.0:1789760713.830480:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692854813798 time 692898340544 state 0x0 ref 1 00000100:00100000:0.0:1789760713.830493:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692898600960 time 692898340544 state 0x1 ref 1 00000100:00100000:0.0:1789760713.830516:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692898600960 time 692935226731 state 0x0 ref 1 00000100:00100000:0.0:1789760713.830526:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508260736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760713.830567:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18665472, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:0.0:1789760713.830575:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.830589:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760713.830593:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.830599:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.830612:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.875994:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971877 is committed 00002000:00010000:0.0:1789760713.876114:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18665472 -> 18669568 00010000:00010000:0.0:1789760713.876129:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.876142:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.876156:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.876163:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760713.876168:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.876177:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.876194:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.876201:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18665472->18669567] (req 18665472->18669567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.876397:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508260736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45866us (46052us total) trans 4294971877 rc 0/0 00000100:00100000:0.0:1789760713.876406:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4589 00000100:00100000:0.0:1789760713.876415:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692898600960 time 692935226731 state 0x2 ref 0 00000100:00100000:1.0:1789760713.879426:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760713.879605:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508260864 00000100:00100000:0.0:1789760713.879616:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508260864 00000100:00100000:0.0:1789760713.879681:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692898600960 time 692935226731 state 0x0 ref 1 00000100:00100000:0.0:1789760713.879696:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692935487147 time 692935226731 state 0x1 ref 1 00000100:00100000:0.0:1789760713.879727:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692935487147 time 692984433914 state 0x0 ref 1 00000100:00100000:0.0:1789760713.879740:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508260864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760713.879790:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18669568, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:0.0:1789760713.879802:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.879820:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=692s v=(0 0) 00010000:00010000:0.0:1789760713.879825:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.879832:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.879848:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760713.909988:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971878 is committed 00002000:00010000:1.0:1789760713.910122:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18669568 -> 18673664 00010000:00010000:1.0:1789760713.910137:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.910149:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.910158:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.910166:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760713.910170:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.910182:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.910273:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.910392:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18669568->18673663] (req 18669568->18673663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.910515:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508260864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30772us (31094us total) trans 4294971878 rc 0/0 00000100:00100000:1.0:1789760713.910526:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4590 00000100:00100000:1.0:1789760713.910538:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692935487147 time 692984433914 state 0x2 ref 0 00000100:00100000:2.0:1789760713.914863:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760713.914975:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508260992 00000100:00100000:0.0:1789760713.914985:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508260992 00000100:00100000:0.0:1789760713.915038:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692935487147 time 692984433914 state 0x0 ref 1 00000100:00100000:0.0:1789760713.915053:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692984694330 time 692984433914 state 0x1 ref 1 00000100:00100000:0.0:1789760713.915115:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692984694330 time 693019825369 state 0x0 ref 1 00000100:00100000:0.0:1789760713.915128:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508260992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760713.915180:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18673664, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:0.0:1789760713.915189:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.915216:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760713.915220:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.915226:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.915240:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760713.941940:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971879 is committed 00002000:00010000:0.0:1789760713.942055:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18673664 -> 18677760 00010000:00010000:0.0:1789760713.942070:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.942081:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.942091:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.942097:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760713.942102:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.942112:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.942134:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.942142:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18673664->18677759] (req 18673664->18677759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760713.942435:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508260992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27303us (27576us total) trans 4294971879 rc 0/0 00000100:00100000:0.0:1789760713.942445:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4591 00000100:00100000:0.0:1789760713.942456:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692984694330 time 693019825369 state 0x2 ref 0 00000100:00100000:0.0:1789760713.945850:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508261120 00000100:00100000:0.0:1789760713.945859:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508261120 00000100:00100000:0.0:1789760713.945908:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 692984694330 time 693019825369 state 0x0 ref 1 00000100:00100000:0.0:1789760713.945924:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693020085785 time 693019825369 state 0x1 ref 1 00000100:00100000:0.0:1789760713.945993:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693020085785 time 693050702791 state 0x0 ref 1 00000100:00100000:0.0:1789760713.946005:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508261120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760713.946051:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18677760, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:0.0:1789760713.946060:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.946075:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760713.946080:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760713.946087:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760713.946100:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760713.968940:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971880 is committed 00002000:00010000:1.0:1789760713.969044:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18677760 -> 18681856 00010000:00010000:1.0:1789760713.969060:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.969072:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.969080:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.969087:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760713.969091:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.969112:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.969132:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.969139:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe543bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18677760->18681855] (req 18677760->18681855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760713.969238:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508261120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23229us (23543us total) trans 4294971880 rc 0/0 00000100:00100000:1.0:1789760713.969248:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4592 00000100:00100000:1.0:1789760713.969259:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693020085785 time 693050702791 state 0x2 ref 0 00000100:00100000:2.0:1789760713.974372:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760713.974826:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508261248 00000100:00100000:1.0:1789760713.974835:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508261248 00000100:00100000:1.0:1789760713.974889:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693020085785 time 693050702791 state 0x0 ref 1 00000100:00100000:1.0:1789760713.974904:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693050963207 time 693050702791 state 0x1 ref 1 00000100:00100000:1.0:1789760713.974935:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693050963207 time 693079644889 state 0x0 ref 1 00000100:00100000:1.0:1789760713.974947:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508261248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760713.974994:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18681856, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:1.0:1789760713.975003:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.975019:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760713.975023:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760713.975029:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760713.975042:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760714.003649:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18681856 -> 18685952 00010000:00010000:1.0:1789760714.003668:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.003681:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.003689:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.003697:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760714.003701:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.003711:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.003729:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.003736:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18681856->18685951] (req 18681856->18685951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.003826:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508261248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28876us (29458us total) trans 4294971881 rc 0/0 00000100:00100000:1.0:1789760714.003840:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4593 00000100:00100000:1.0:1789760714.003849:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693050963207 time 693079644889 state 0x2 ref 0 00000001:00080000:1.0:1789760714.003884:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971881 is committed 00000100:00100000:1.0:1789760714.007696:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508261376 00000100:00100000:1.0:1789760714.007706:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508261376 00000100:00100000:1.0:1789760714.007759:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693050963207 time 693079644889 state 0x0 ref 1 00000100:00100000:1.0:1789760714.007774:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693079905305 time 693079644889 state 0x1 ref 1 00000100:00100000:1.0:1789760714.007797:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693079905305 time 693112507790 state 0x0 ref 1 00000100:00100000:1.0:1789760714.007809:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508261376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760714.007857:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18685952, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:1.0:1789760714.007867:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.007883:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.007888:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.007893:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.007908:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.043736:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971882 is committed 00002000:00010000:1.0:1789760714.043806:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18685952 -> 18690048 00010000:00010000:1.0:1789760714.043820:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.043833:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.043842:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.043849:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760714.043854:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.043863:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.043883:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.043890:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18685952->18690047] (req 18685952->18690047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.043982:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508261376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36169us (36495us total) trans 4294971882 rc 0/0 00000100:00100000:1.0:1789760714.043992:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4594 00000100:00100000:1.0:1789760714.044002:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693079905305 time 693112507790 state 0x2 ref 0 00000100:00100000:0.0:1789760714.051331:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508261504 00000100:00100000:0.0:1789760714.051341:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508261504 00000100:00100000:0.0:1789760714.051396:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693079905305 time 693112507790 state 0x0 ref 1 00000100:00100000:0.0:1789760714.051410:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693112768206 time 693112507790 state 0x1 ref 1 00000100:00100000:0.0:1789760714.051430:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693112768206 time 693156141418 state 0x0 ref 1 00000100:00100000:0.0:1789760714.051442:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508261504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760714.051487:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18690048, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:0.0:1789760714.051496:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.051510:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760714.051514:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.051520:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.051534:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760714.117525:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18690048 -> 18694144 00010000:00010000:1.0:1789760714.117546:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.117558:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.117567:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.117574:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760714.117579:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.117588:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.117606:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.117614:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe543d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18690048->18694143] (req 18690048->18694143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.117827:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508261504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 66380us (67233us total) trans 4294971883 rc 0/0 00000100:00100000:1.0:1789760714.117837:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4595 00000100:00100000:1.0:1789760714.117848:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693112768206 time 693156141418 state 0x2 ref 0 00000100:00100000:1.0:1789760714.120919:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760714.121004:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508261632 00000100:00100000:2.0:1789760714.121013:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508261632 00000100:00100000:2.0:1789760714.121065:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693112768206 time 693156141418 state 0x0 ref 1 00000100:00100000:2.0:1789760714.121083:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693156401834 time 693156141418 state 0x1 ref 1 00000100:00100000:2.0:1789760714.121114:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693156401834 time 693225824507 state 0x0 ref 1 00000100:00100000:2.0:1789760714.121126:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508261632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760714.121172:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18694144, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:2.0:1789760714.121181:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe543db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.121262:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.121268:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.121275:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe543db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.121330:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe543db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760714.148189:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18694144 -> 18698240 00010000:00010000:2.0:1789760714.148213:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe543db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.148225:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe543db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.148238:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe543db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.148252:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760714.148257:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe543db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.148267:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe543db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.148315:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe543db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.148324:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe543db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18694144->18698239] (req 18694144->18698239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.148423:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508261632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27292us (27506us total) trans 4294971884 rc 0/0 00000100:00100000:2.0:1789760714.148436:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4596 00000100:00100000:2.0:1789760714.148447:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693156401834 time 693225824507 state 0x2 ref 0 00000100:00100000:2.0:1789760714.152372:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760714.152640:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508261760 00000100:00100000:1.0:1789760714.152650:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508261760 00000100:00100000:1.0:1789760714.152703:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693156401834 time 693225824507 state 0x0 ref 1 00000100:00100000:1.0:1789760714.152718:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693226084923 time 693225824507 state 0x1 ref 1 00000100:00100000:1.0:1789760714.152747:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693226084923 time 693257457650 state 0x0 ref 1 00000100:00100000:1.0:1789760714.152760:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508261760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760714.152806:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18698240, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:1.0:1789760714.152815:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.152830:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.152835:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.152842:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.152857:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.217372:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971885 is committed 00002000:00010000:1.0:1789760714.217482:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18698240 -> 18702336 00010000:00010000:1.0:1789760714.217497:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.217517:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.217527:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.217535:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760714.217540:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.217754:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.217779:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.217846:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18698240->18702335] (req 18698240->18702335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.217943:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508261760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 65179us (65576us total) trans 4294971885 rc 0/0 00000100:00100000:1.0:1789760714.217953:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4597 00000100:00100000:1.0:1789760714.217962:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693226084923 time 693257457650 state 0x2 ref 0 00000100:00100000:0.0:1789760714.225977:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760714.226065:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508261888 00000100:00100000:1.0:1789760714.226074:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508261888 00000100:00100000:1.0:1789760714.226125:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693226084923 time 693257457650 state 0x0 ref 1 00000100:00100000:1.0:1789760714.226139:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693257718066 time 693257457650 state 0x1 ref 1 00000100:00100000:1.0:1789760714.226166:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693257718066 time 693330877009 state 0x0 ref 1 00000100:00100000:1.0:1789760714.226176:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508261888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760714.226231:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18702336, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:1.0:1789760714.226239:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.226255:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.226259:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.226265:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.226320:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760714.264240:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18702336 -> 18706432 00010000:00010000:1.0:1789760714.264258:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.264271:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.264324:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.264332:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760714.264336:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.264345:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.264365:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.264373:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18702336->18706431] (req 18702336->18706431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.264454:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508261888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38274us (38482us total) trans 4294971886 rc 0/0 00000100:00100000:1.0:1789760714.265217:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4598 00000100:00100000:1.0:1789760714.265226:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693257718066 time 693330877009 state 0x2 ref 0 00000001:00080000:0.0:1789760714.268356:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971886 is committed 00000100:00100000:0.0:1789760714.269403:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760714.269711:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508262016 00000100:00100000:2.0:1789760714.269786:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508262016 00000100:00100000:2.0:1789760714.269845:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693257718066 time 693330877009 state 0x0 ref 1 00000100:00100000:2.0:1789760714.269860:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693331137425 time 693330877009 state 0x1 ref 1 00000100:00100000:2.0:1789760714.269932:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693331137425 time 693374642772 state 0x0 ref 1 00000100:00100000:2.0:1789760714.269946:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508262016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760714.269992:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18706432, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:2.0:1789760714.270000:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe543f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.270016:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.270020:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.270026:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe543f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.270039:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe543f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760714.313929:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18706432 -> 18710528 00010000:00010000:2.0:1789760714.313948:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe543f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.313960:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe543f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.313968:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe543f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.313974:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760714.313979:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe543f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.313988:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe543f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.314006:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe543f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.314012:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe543f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18706432->18710527] (req 18706432->18710527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.314144:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971887 is committed 00000100:00100000:2.0:1789760714.314196:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508262016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44241us (44787us total) trans 4294971887 rc 0/0 00000100:00100000:2.0:1789760714.314206:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4599 00000100:00100000:2.0:1789760714.314216:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693331137425 time 693374642772 state 0x2 ref 0 00000100:00100000:2.0:1789760714.318523:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760714.318734:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508262144 00000100:00100000:1.0:1789760714.318744:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508262144 00000100:00100000:1.0:1789760714.318794:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693331137425 time 693374642772 state 0x0 ref 1 00000100:00100000:1.0:1789760714.318809:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693374903188 time 693374642772 state 0x1 ref 1 00000100:00100000:1.0:1789760714.318839:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693374903188 time 693423549432 state 0x0 ref 1 00000100:00100000:1.0:1789760714.318851:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508262144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760714.318897:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18710528, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:1.0:1789760714.318905:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.318921:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.318925:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.318932:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.318945:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760714.370188:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18710528 -> 18714624 00010000:00010000:2.0:1789760714.370202:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.370216:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.370225:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.370238:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760714.370243:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.370251:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.370270:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.370303:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe543f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18710528->18714623] (req 18710528->18714623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.370390:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508262144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51536us (51871us total) trans 4294971888 rc 0/0 00000100:00100000:2.0:1789760714.370400:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4600 00000100:00100000:2.0:1789760714.370409:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693374903188 time 693423549432 state 0x2 ref 0 00000100:00100000:1.0:1789760714.378446:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508262272 00000100:00100000:1.0:1789760714.378456:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508262272 00000100:00100000:1.0:1789760714.378509:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693374903188 time 693423549432 state 0x0 ref 1 00000100:00100000:1.0:1789760714.378525:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693423809848 time 693423549432 state 0x1 ref 1 00000100:00100000:1.0:1789760714.378546:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693423809848 time 693483256645 state 0x0 ref 1 00000100:00100000:1.0:1789760714.378559:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508262272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760714.378609:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18714624, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:1.0:1789760714.378619:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.378634:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.378706:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.378714:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.378730:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760714.427093:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18714624 -> 18718720 00000001:00080000:0.0:1789760714.427098:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971889 is committed 00010000:00010000:1.0:1789760714.427113:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.427126:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.427135:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.427142:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760714.427147:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.427156:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.427183:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.427191:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe543fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18714624->18718719] (req 18714624->18718719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.427329:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508262272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48766us (48989us total) trans 4294971889 rc 0/0 00000100:00100000:1.0:1789760714.427340:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4601 00000100:00100000:1.0:1789760714.427349:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693423809848 time 693483256645 state 0x2 ref 0 00000100:00100000:0.0:1789760714.434063:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760714.434150:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508262400 00000100:00100000:2.0:1789760714.434161:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508262400 00000100:00100000:2.0:1789760714.434227:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693423809848 time 693483256645 state 0x0 ref 1 00000100:00100000:2.0:1789760714.434242:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693483517061 time 693483256645 state 0x1 ref 1 00000100:00100000:2.0:1789760714.434724:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693483517061 time 693539431991 state 0x0 ref 1 00000100:00100000:2.0:1789760714.434743:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508262400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760714.434787:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18718720, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:2.0:1789760714.434795:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54405 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.434810:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.434814:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.434821:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54405 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.434834:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54405 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760714.473302:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971890 is committed 00002000:00010000:2.0:1789760714.473365:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18718720 -> 18722816 00010000:00010000:2.0:1789760714.473382:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54405 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.473397:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54405 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.473406:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54405 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.473414:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760714.473419:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54405 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.473429:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54405 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.473450:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54405 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.473458:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54405 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18718720->18722815] (req 18718720->18722815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.473566:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508262400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38821us (39514us total) trans 4294971890 rc 0/0 00000100:00100000:2.0:1789760714.473578:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4602 00000100:00100000:2.0:1789760714.473587:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693483517061 time 693539431991 state 0x2 ref 0 00000100:00100000:0.0:1789760714.477974:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760714.478018:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508262528 00000100:00100000:2.0:1789760714.478032:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508262528 00000100:00100000:2.0:1789760714.478160:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693483517061 time 693539431991 state 0x0 ref 1 00000100:00100000:2.0:1789760714.478175:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693539692407 time 693539431991 state 0x1 ref 1 00000100:00100000:2.0:1789760714.478205:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693539692407 time 693582915806 state 0x0 ref 1 00000100:00100000:2.0:1789760714.478217:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508262528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760714.478283:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18722816, mtime: 0x6aad94ca, atime: 0x0, ctime: 0x6aad94ca, blocks: 0x9007 00010000:00010000:2.0:1789760714.478327:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5440c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.478343:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.478347:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.478353:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5440c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.478365:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5440c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.495740:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760714.514128:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971891 is committed 00002000:00010000:2.0:1789760714.514427:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18722816 -> 18726912 00002000:00010000:2.0:1789760714.514432:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760714 -> 1789760715 00002000:00010000:2.0:1789760714.514435:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760714 -> 1789760715 00010000:00010000:2.0:1789760714.514447:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5440c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.514460:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5440c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.514470:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5440c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.514478:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760714.514483:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5440c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.514491:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5440c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.514512:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5440c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.514521:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5440c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18722816->18726911] (req 18722816->18726911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.514615:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508262528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36395us (36643us total) trans 4294971891 rc 0/0 00000100:00100000:2.0:1789760714.514625:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4603 00000100:00100000:2.0:1789760714.514634:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693539692407 time 693582915806 state 0x2 ref 0 00000100:00100000:1.0:1789760714.520551:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508262656 00000100:00100000:1.0:1789760714.520560:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508262656 00000100:00100000:1.0:1789760714.520622:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693539692407 time 693582915806 state 0x0 ref 1 00000100:00100000:1.0:1789760714.520646:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693583176222 time 693582915806 state 0x1 ref 1 00000100:00100000:1.0:1789760714.520672:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693583176222 time 693625383141 state 0x0 ref 1 00000100:00100000:1.0:1789760714.520684:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508262656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760714.520732:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18726912, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:1.0:1789760714.520741:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54413 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.520757:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.520762:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.520768:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54413 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.520782:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54413 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.543137:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971892 is committed 00002000:00010000:1.0:1789760714.543234:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18726912 -> 18731008 00010000:00010000:1.0:1789760714.543247:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54413 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.543259:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54413 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.543313:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54413 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.543321:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760714.543325:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54413 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.543334:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54413 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.543352:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54413 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.543359:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54413 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18726912->18731007] (req 18726912->18731007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.543441:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508262656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22754us (23163us total) trans 4294971892 rc 0/0 00000100:00100000:1.0:1789760714.543451:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4604 00000100:00100000:1.0:1789760714.543462:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693583176222 time 693625383141 state 0x2 ref 0 00000100:00100000:0.0:1789760714.547741:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760714.547828:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508262784 00000100:00100000:2.0:1789760714.547837:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508262784 00000100:00100000:2.0:1789760714.547885:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693583176222 time 693625383141 state 0x0 ref 1 00000100:00100000:2.0:1789760714.547900:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693625643557 time 693625383141 state 0x1 ref 1 00000100:00100000:2.0:1789760714.547930:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693625643557 time 693652640157 state 0x0 ref 1 00000100:00100000:2.0:1789760714.547941:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508262784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760714.547986:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18731008, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:2.0:1789760714.547995:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5441a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.548014:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.548018:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.548025:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5441a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.548038:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5441a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18731008->18735103] (req 18731008->18735103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.575410:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971893 is committed 00000100:00100000:2.0:1789760714.579920:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760714.580016:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508262912 00000100:00100000:1.0:1789760714.580027:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508262912 00000100:00100000:1.0:1789760714.580080:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693625643557 time 693652640157 state 0x0 ref 1 00000100:00100000:1.0:1789760714.580096:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693652900573 time 693652640157 state 0x1 ref 1 00000100:00100000:1.0:1789760714.580129:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693652900573 time 693684839610 state 0x0 ref 1 00000100:00100000:1.0:1789760714.580141:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508262912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760714.580188:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18735104, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:1.0:1789760714.580197:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54421 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.580213:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.580217:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.580223:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54421 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.580238:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54421 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.631448:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971894 is committed 00002000:00010000:2.0:1789760714.631917:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18735104 -> 18739200 00010000:00010000:2.0:1789760714.631931:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54421 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.631943:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54421 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.631951:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54421 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.631958:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760714.631962:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54421 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.631971:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54421 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.631996:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54421 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.632003:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54421 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18735104->18739199] (req 18735104->18739199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.632089:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508262912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51946us (52174us total) trans 4294971894 rc 0/0 00000100:00100000:2.0:1789760714.632100:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4606 00000100:00100000:2.0:1789760714.632109:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693652900573 time 693684839610 state 0x2 ref 0 00000100:00100000:1.0:1789760714.640810:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508263040 00000100:00100000:1.0:1789760714.640823:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508263040 00000100:00100000:1.0:1789760714.640887:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693652900573 time 693684839610 state 0x0 ref 1 00000100:00100000:1.0:1789760714.640903:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693685100026 time 693684839610 state 0x1 ref 1 00000100:00100000:1.0:1789760714.640933:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693685100026 time 693745643086 state 0x0 ref 1 00000100:00100000:1.0:1789760714.640946:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508263040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760714.640994:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18739200, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:1.0:1789760714.641002:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54428 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.641017:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.641021:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.641028:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54428 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.641041:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54428 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760714.677420:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18739200 -> 18743296 00010000:00010000:1.0:1789760714.677441:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54428 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.677460:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54428 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.677468:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54428 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.677475:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760714.677480:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54428 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.677489:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54428 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.677510:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54428 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.677519:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54428 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18739200->18743295] (req 18739200->18743295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760714.677621:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508263040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36672us (37228us total) trans 4294971895 rc 0/0 00000100:00100000:1.0:1789760714.677639:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4607 00000100:00100000:1.0:1789760714.677649:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693685100026 time 693745643086 state 0x2 ref 0 00000100:00100000:0.0:1789760714.684342:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760714.723175:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971896 is committed 00002000:00010000:1.0:1789760714.723258:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18743296 -> 18747392 00010000:00010000:1.0:1789760714.723283:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5442f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.723425:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5442f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.723435:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5442f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.723443:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760714.723448:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5442f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.723458:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5442f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.723478:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5442f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.723486:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5442f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18743296->18747391] (req 18743296->18747391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.723761:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508263168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38939us (39479us total) trans 4294971896 rc 0/0 00000100:00100000:0.0:1789760714.723773:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4608 00000100:00100000:0.0:1789760714.723783:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693745903502 time 693789512015 state 0x2 ref 0 00000100:00100000:0.0:1789760714.729400:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760714.729676:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508263296 00000100:00100000:2.0:1789760714.729688:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508263296 00000100:00100000:2.0:1789760714.729750:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693745903502 time 693789512015 state 0x0 ref 1 00000100:00100000:2.0:1789760714.729767:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693789772431 time 693789512015 state 0x1 ref 1 00000100:00100000:2.0:1789760714.729800:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693789772431 time 693834510363 state 0x0 ref 1 00000100:00100000:2.0:1789760714.729814:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508263296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760714.729870:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18747392, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:2.0:1789760714.729880:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54436 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.729898:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.729903:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.729911:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54436 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.729926:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54436 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.767315:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971897 is committed 00002000:00010000:2.0:1789760714.767411:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18747392 -> 18751488 00010000:00010000:2.0:1789760714.767427:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54436 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.767441:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54436 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.767450:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54436 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.767457:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760714.767462:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54436 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.767471:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54436 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.767489:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54436 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.767497:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54436 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18747392->18751487] (req 18747392->18751487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760714.767586:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508263296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37770us (38193us total) trans 4294971897 rc 0/0 00000100:00100000:2.0:1789760714.767596:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4609 00000100:00100000:2.0:1789760714.767606:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693789772431 time 693834510363 state 0x2 ref 0 00000100:00100000:1.0:1789760714.771949:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508263424 00000100:00100000:1.0:1789760714.771959:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508263424 00000100:00100000:1.0:1789760714.772012:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693789772431 time 693834510363 state 0x0 ref 1 00000100:00100000:1.0:1789760714.772023:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693834770779 time 693834510363 state 0x1 ref 1 00000100:00100000:1.0:1789760714.772045:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693834770779 time 693876755775 state 0x0 ref 1 00000100:00100000:1.0:1789760714.772056:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508263424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760714.772100:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18751488, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:1.0:1789760714.772113:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5443d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.772127:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:1.0:1789760714.772132:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.772138:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5443d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.772157:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5443d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760714.800253:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971898 is committed 00002000:00010000:0.0:1789760714.800410:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18751488 -> 18755584 00010000:00010000:0.0:1789760714.800425:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5443d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.800438:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5443d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.800454:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5443d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.800461:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760714.800466:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5443d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.800475:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5443d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.800495:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5443d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.800504:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5443d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18751488->18755583] (req 18751488->18755583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.801023:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508263424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28963us (29173us total) trans 4294971898 rc 0/0 00000100:00100000:0.0:1789760714.801043:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4610 00000100:00100000:0.0:1789760714.801054:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693834770779 time 693876755775 state 0x2 ref 0 00000100:00100000:0.0:1789760714.804104:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760714.817716:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760714.817863:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760714.817900:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760714.817923:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760714.817960:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760714.818477:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508263936 00000100:00100000:1.0:1789760714.818482:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508263936 00000100:00100000:1.0:1789760714.818523:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508264192 00000100:00100000:1.0:1789760714.818526:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508264192 00000100:00100000:1.0:1789760714.818542:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508263808 00000100:00100000:1.0:1789760714.818545:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508263808 00000100:00100000:1.0:1789760714.818559:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508264064 00000100:00100000:1.0:1789760714.818561:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508264064 00000100:00100000:1.0:1789760714.818578:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 839 00000100:00100000:1.0:1789760714.818585:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508263936:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760714.818651:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508263936:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 66us (788us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760714.818661:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 839 00000100:00100000:1.0:1789760714.818673:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 840 00000100:00100000:1.0:1789760714.818678:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508264192:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760714.818704:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508264192:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 25us (804us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760714.818710:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 840 00000100:00100000:1.0:1789760714.818723:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 841 00000100:00100000:1.0:1789760714.818730:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508263808:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760714.818753:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508263808:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 24us (830us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760714.818759:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 841 00000100:00100000:1.0:1789760714.818769:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 842 00000100:00100000:1.0:1789760714.818775:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508264064:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760714.818795:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508264064:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 20us (836us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760714.818800:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 842 00000100:00080000:2.0:1789760714.818899:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760714.818904:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000001:00080000:2.0:1789760714.823533:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971899 is committed 00002000:00010000:0.0:1789760714.823718:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18755584 -> 18759680 00010000:00010000:0.0:1789760714.823734:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54444 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.823747:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54444 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.823755:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54444 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.823762:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760714.823767:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54444 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.823776:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54444 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.823869:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54444 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.823878:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54444 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18755584->18759679] (req 18755584->18759679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.824036:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508263552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19270us (19936us total) trans 4294971899 rc 0/0 00000100:00100000:0.0:1789760714.824050:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4611 00000100:00100000:0.0:1789760714.824060:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693877016191 time 693909458717 state 0x2 ref 0 00000100:00100000:0.0:1789760714.827589:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760714.827659:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508264320 00000100:00100000:2.0:1789760714.827670:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508264320 00000100:00100000:2.0:1789760714.827789:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693877016191 time 693909458717 state 0x0 ref 1 00000100:00100000:2.0:1789760714.827804:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693909719133 time 693909458717 state 0x1 ref 1 00000100:00100000:2.0:1789760714.827833:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693909719133 time 693932543883 state 0x0 ref 1 00000100:00100000:2.0:1789760714.827844:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508264320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760714.827893:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18759680, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:2.0:1789760714.827902:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5444b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.827917:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:2.0:1789760714.827921:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760714.827928:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5444b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760714.827944:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5444b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760714.876264:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18759680 -> 18763776 00010000:00010000:0.0:1789760714.876302:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5444b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.876315:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5444b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.876325:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5444b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.876332:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760714.876337:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5444b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.876345:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5444b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.876366:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5444b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.876375:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5444b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18759680->18763775] (req 18759680->18763775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.876472:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508264320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48624us (48887us total) trans 4294971900 rc 0/0 00000100:00100000:0.0:1789760714.876483:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4612 00000100:00100000:0.0:1789760714.876492:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693909719133 time 693932543883 state 0x2 ref 0 00000100:00100000:0.0:1789760714.880328:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508264448 00000100:00100000:0.0:1789760714.880338:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508264448 00000100:00100000:0.0:1789760714.880400:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693909719133 time 693932543883 state 0x0 ref 1 00000100:00100000:0.0:1789760714.880414:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693932804299 time 693932543883 state 0x1 ref 1 00000100:00100000:0.0:1789760714.880453:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693932804299 time 693985159433 state 0x0 ref 1 00000100:00100000:0.0:1789760714.880466:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508264448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760714.880512:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18763776, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:0.0:1789760714.880521:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54452 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.880537:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=693s v=(0 0) 00010000:00010000:0.0:1789760714.880541:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.880547:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54452 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.880561:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54452 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760714.905348:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18763776 -> 18767872 00010000:00010000:0.0:1789760714.905363:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54452 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.905376:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54452 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.905384:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54452 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.905391:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760714.905395:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54452 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.905404:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54452 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.905426:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54452 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.905434:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54452 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18763776->18767871] (req 18763776->18767871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.905565:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508264448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25097us (25669us total) trans 4294971901 rc 0/0 00000100:00100000:0.0:1789760714.905578:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4613 00000100:00100000:0.0:1789760714.905587:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693932804299 time 693985159433 state 0x2 ref 0 00000100:00100000:1.0:1789760714.909050:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760714.909584:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508264576 00000100:00100000:0.0:1789760714.909593:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508264576 00000100:00100000:0.0:1789760714.909701:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693932804299 time 693985159433 state 0x0 ref 1 00000100:00100000:0.0:1789760714.909715:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693985419849 time 693985159433 state 0x1 ref 1 00000100:00100000:0.0:1789760714.909784:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693985419849 time 694014493788 state 0x0 ref 1 00000100:00100000:0.0:1789760714.909796:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508264576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760714.909842:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18767872, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:0.0:1789760714.909852:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54459 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.909867:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760714.909871:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760714.909877:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54459 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.909891:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54459 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760714.937904:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971902 is committed 00002000:00010000:0.0:1789760714.937989:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18767872 -> 18771968 00010000:00010000:0.0:1789760714.938006:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54459 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.938020:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54459 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.938029:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54459 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.938036:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760714.938041:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54459 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.938050:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54459 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.938070:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54459 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.938078:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54459 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18767872->18771967] (req 18767872->18771967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.938335:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508264576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28536us (29287us total) trans 4294971902 rc 0/0 00000100:00100000:0.0:1789760714.938347:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4614 00000100:00100000:0.0:1789760714.938357:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693985419849 time 694014493788 state 0x2 ref 0 00000100:00100000:0.0:1789760714.941725:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760714.941796:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508264704 00000100:00100000:1.0:1789760714.941806:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508264704 00000100:00100000:1.0:1789760714.941859:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 693985419849 time 694014493788 state 0x0 ref 1 00000100:00100000:1.0:1789760714.941875:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694014754204 time 694014493788 state 0x1 ref 1 00000100:00100000:1.0:1789760714.941910:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694014754204 time 694046619859 state 0x0 ref 1 00000100:00100000:1.0:1789760714.941922:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508264704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760714.941974:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18771968, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:1.0:1789760714.941983:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54460 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.942000:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:1.0:1789760714.942004:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760714.942010:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54460 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760714.942027:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54460 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760714.969408:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18771968 -> 18776064 00010000:00010000:0.0:1789760714.969427:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54460 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.969439:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54460 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.969447:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54460 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.969454:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760714.969458:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54460 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.969467:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54460 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.969486:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54460 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760714.969493:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54460 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18771968->18776063] (req 18771968->18776063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760714.969583:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508264704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27657us (27862us total) trans 4294971903 rc 0/0 00000100:00100000:0.0:1789760714.969595:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4615 00000100:00100000:0.0:1789760714.969604:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694014754204 time 694046619859 state 0x2 ref 0 00000001:00080000:2.0:1789760714.969736:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971903 is committed 00000100:00100000:2.0:1789760714.973172:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760715.000546:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971904 is committed 00002000:00010000:2.0:1789760715.000816:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18776064 -> 18780160 00010000:00010000:2.0:1789760715.000831:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54467 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.000845:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54467 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.000854:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54467 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.000860:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760715.000865:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54467 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.000874:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54467 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.000893:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54467 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.000904:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54467 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18776064->18780159] (req 18776064->18780159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.000995:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508264832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27494us (27829us total) trans 4294971904 rc 0/0 00000100:00100000:2.0:1789760715.001006:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4616 00000100:00100000:2.0:1789760715.001014:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694046880275 time 694078196942 state 0x2 ref 0 00000100:00100000:1.0:1789760715.004154:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760715.004279:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508264960 00000100:00100000:0.0:1789760715.004314:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508264960 00000100:00100000:0.0:1789760715.004373:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694046880275 time 694078196942 state 0x0 ref 1 00000100:00100000:0.0:1789760715.004389:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694078457358 time 694078196942 state 0x1 ref 1 00000100:00100000:0.0:1789760715.004411:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694078457358 time 694109122264 state 0x0 ref 1 00000100:00100000:0.0:1789760715.004423:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508264960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760715.004469:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18780160, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:0.0:1789760715.004478:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5446e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.004492:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760715.004497:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.004503:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5446e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18780160->18784255] (req 18780160->18784255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.004519:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5446e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18780160->18784255] (req 18780160->18784255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.037714:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760715.037803:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508265088 00000100:00100000:2.0:1789760715.037813:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508265088 00000100:00100000:2.0:1789760715.037866:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694078457358 time 694109122264 state 0x0 ref 1 00000100:00100000:2.0:1789760715.037882:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694109382680 time 694109122264 state 0x1 ref 1 00000100:00100000:2.0:1789760715.037912:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694109382680 time 694142622312 state 0x0 ref 1 00000100:00100000:2.0:1789760715.037925:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508265088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760715.037971:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18784256, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:2.0:1789760715.037980:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54475 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.037995:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760715.038000:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.038005:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54475 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.038025:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54475 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18784256->18788351] (req 18784256->18788351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.039413:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971905 is committed 00000100:00100000:0.0:1789760715.071577:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508265216 00000100:00100000:0.0:1789760715.071586:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508265216 00000100:00100000:0.0:1789760715.071696:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694109382680 time 694142622312 state 0x0 ref 1 00000100:00100000:0.0:1789760715.071711:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694142882728 time 694142622312 state 0x1 ref 1 00000100:00100000:0.0:1789760715.071741:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694142882728 time 694176450993 state 0x0 ref 1 00000100:00100000:0.0:1789760715.071753:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfeed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508265216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760715.071799:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18788352, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:0.0:1789760715.071808:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5447c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.071824:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760715.071828:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.071834:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5447c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.071847:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5447c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18788352->18792447] (req 18788352->18792447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.097111:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971907 is committed 00000100:00100000:0.0:1789760715.101528:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760715.132329:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971908 is committed 00000100:00100000:2.0:1789760715.135565:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:1.0:1789760715.163461:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971909 is committed 00000100:00100000:1.0:1789760715.168084:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760715.194634:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971910 is committed 00000100:00100000:0.0:1789760715.197904:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508265728 00000100:00100000:0.0:1789760715.197914:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508265728 00000100:00100000:0.0:1789760715.197967:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694240724924 time 694273252889 state 0x0 ref 1 00000100:00100000:0.0:1789760715.197981:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694273513305 time 694273252889 state 0x1 ref 1 00000100:00100000:0.0:1789760715.198014:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694273513305 time 694302724753 state 0x0 ref 1 00000100:00100000:0.0:1789760715.198029:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508265728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760715.198076:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18804736, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:0.0:1789760715.198086:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54498 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.198102:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760715.198106:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.198112:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54498 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.198126:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54498 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760715.227649:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18804736 -> 18808832 00010000:00010000:0.0:1789760715.227670:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54498 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.227684:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54498 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.227693:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54498 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.227699:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760715.227703:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54498 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.227712:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54498 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.227731:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54498 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.227739:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54498 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18804736->18808831] (req 18804736->18808831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.227834:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508265728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29802us (30023us total) trans 4294971911 rc 0/0 00000100:00100000:0.0:1789760715.227844:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4623 00000100:00100000:0.0:1789760715.227854:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694273513305 time 694302724753 state 0x2 ref 0 00000100:00100000:2.0:1789760715.232070:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:1.0:1789760715.275999:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971912 is committed 00002000:00010000:2.0:1789760715.276100:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18808832 -> 18812928 00010000:00010000:2.0:1789760715.276115:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5449f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.276128:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5449f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.276138:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5449f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.276146:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760715.276150:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5449f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.276160:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5449f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.276251:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5449f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.276283:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5449f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18808832->18812927] (req 18808832->18812927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.276397:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508265856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43568us (44331us total) trans 4294971912 rc 0/0 00000100:00100000:2.0:1789760715.276408:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4624 00000100:00100000:2.0:1789760715.276417:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694302985169 time 694337523252 state 0x2 ref 0 00000100:00100000:2.0:1789760715.281141:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:1.0:1789760715.326108:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971913 is committed 00000100:00100000:0.0:1789760715.329999:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508266112 00000100:00100000:0.0:1789760715.330011:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508266112 00000100:00100000:0.0:1789760715.330066:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694337783668 time 694386117086 state 0x0 ref 1 00000100:00100000:0.0:1789760715.330080:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694386377502 time 694386117086 state 0x1 ref 1 00000100:00100000:0.0:1789760715.330114:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694386377502 time 694434824323 state 0x0 ref 1 00000100:00100000:0.0:1789760715.330125:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508266112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760715.330171:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18817024, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:0.0:1789760715.330179:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe544ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.330203:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760715.330208:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.330214:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe544ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.330231:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe544ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760715.358843:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971914 is committed 00002000:00010000:2.0:1789760715.359891:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18817024 -> 18821120 00010000:00010000:2.0:1789760715.359910:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe544ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.359923:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe544ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.359931:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe544ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.359938:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760715.359942:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe544ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.359951:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe544ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.359976:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe544ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.359984:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe544ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18817024->18821119] (req 18817024->18821119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.360094:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508266112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29964us (30195us total) trans 4294971914 rc 0/0 00000100:00100000:2.0:1789760715.360105:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4626 00000100:00100000:2.0:1789760715.360115:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694386377502 time 694434824323 state 0x2 ref 0 00000100:00100000:0.0:1789760715.365568:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760715.365645:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508266240 00000100:00100000:2.0:1789760715.365655:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508266240 00000100:00100000:2.0:1789760715.365706:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694386377502 time 694434824323 state 0x0 ref 1 00000100:00100000:2.0:1789760715.365721:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694435084739 time 694434824323 state 0x1 ref 1 00000100:00100000:2.0:1789760715.365748:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694435084739 time 694470459486 state 0x0 ref 1 00000100:00100000:2.0:1789760715.365759:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508266240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760715.365805:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18821120, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:2.0:1789760715.365813:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.365829:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760715.365834:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.365841:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.365855:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760715.403537:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18821120 -> 18825216 00010000:00010000:2.0:1789760715.403552:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.403565:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.403575:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.403583:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760715.403588:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.403597:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.403626:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.403635:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18821120->18825215] (req 18821120->18825215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.403771:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508266240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38008us (38206us total) trans 4294971915 rc 0/0 00000100:00100000:2.0:1789760715.403782:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4627 00000100:00100000:2.0:1789760715.403792:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694435084739 time 694470459486 state 0x2 ref 0 00000100:00100000:1.0:1789760715.411610:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760715.475982:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508266496 00000100:00100000:2.0:1789760715.475991:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508266496 00000100:00100000:2.0:1789760715.476047:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694470719902 time 694516566454 state 0x0 ref 1 00000100:00100000:2.0:1789760715.476063:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694516826870 time 694516566454 state 0x1 ref 1 00000100:00100000:2.0:1789760715.476093:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694516826870 time 694580804093 state 0x0 ref 1 00000100:00100000:2.0:1789760715.476104:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508266496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760715.476151:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18829312, mtime: 0x6aad94cb, atime: 0x0, ctime: 0x6aad94cb, blocks: 0x9007 00010000:00010000:2.0:1789760715.476159:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.476175:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760715.476442:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.476450:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.476466:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760715.529917:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18829312 -> 18833408 00002000:00010000:0.0:1789760715.529928:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760715 -> 1789760716 00002000:00010000:0.0:1789760715.529931:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760715 -> 1789760716 00010000:00010000:0.0:1789760715.529944:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.529957:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.529967:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.529975:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760715.529979:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.529988:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.530008:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.530017:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe544c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18829312->18833407] (req 18829312->18833407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.530112:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508266496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54004us (54509us total) trans 4294971917 rc 0/0 00000100:00100000:0.0:1789760715.530123:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4629 00000100:00100000:0.0:1789760715.530132:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694516826870 time 694580804093 state 0x2 ref 0 00000100:00100000:0.0:1789760715.534699:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760715.534790:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508266624 00000100:00100000:2.0:1789760715.534799:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508266624 00000100:00100000:2.0:1789760715.534851:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694516826870 time 694580804093 state 0x0 ref 1 00000100:00100000:2.0:1789760715.534866:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694581064509 time 694580804093 state 0x1 ref 1 00000100:00100000:2.0:1789760715.534945:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694581064509 time 694639654258 state 0x0 ref 1 00000100:00100000:2.0:1789760715.534958:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508266624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760715.535006:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18833408, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9007 00010000:00010000:2.0:1789760715.535015:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe544c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.535030:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760715.535034:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.535040:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe544c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.535054:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe544c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18833408->18837503] (req 18833408->18837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760715.573934:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971918 is committed 00000100:00100000:2.0:1789760715.583713:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760715.614965:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18837504 -> 18841600 00010000:00010000:0.0:1789760715.614984:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe544d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.615006:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe544d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.615015:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe544d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.615023:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760715.615027:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe544d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.615038:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe544d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.615058:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe544d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.615066:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe544d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18837504->18841599] (req 18837504->18841599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.615162:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508266752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30980us (31603us total) trans 4294971919 rc 0/0 00000100:00100000:0.0:1789760715.615172:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4631 00000100:00100000:0.0:1789760715.615182:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694639914674 time 694688875728 state 0x2 ref 0 00000001:00080000:0.0:1789760715.615224:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971919 is committed 00000100:00100000:0.0:1789760715.624602:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760715.624895:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508266880 00000100:00100000:2.0:1789760715.624904:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508266880 00000100:00100000:2.0:1789760715.624967:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694639914674 time 694688875728 state 0x0 ref 1 00000100:00100000:2.0:1789760715.624984:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694689136144 time 694688875728 state 0x1 ref 1 00000100:00100000:2.0:1789760715.625019:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694689136144 time 694729729570 state 0x0 ref 1 00000100:00100000:2.0:1789760715.625032:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508266880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760715.625080:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18841600, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9007 00010000:00010000:2.0:1789760715.625089:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe544d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.625106:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:2.0:1789760715.625110:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.625116:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe544d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.625130:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe544d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.647963:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534057856:0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.648003:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760715.648047:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd99c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3475:1876699534058368:0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.648084:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760715.648366:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04884219500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3475:1876699534058880:0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.648397:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760715.648666:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b644a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534057728:0@lo:400:kworker.0 00000100:00100000:0.0:1789760715.649143:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:0.0:1789760715.649152:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534057856:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.649516:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760715.649526:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534059136:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760715.649600:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534059392:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760715.649639:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760715.649655:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760715.650507:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534059264:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760715.650535:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760715.650561:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2ddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534059392:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760715.650806:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760715.650823:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534059264:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760715.650992:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534057856:0@lo:400:kworker.0 00000100:00100000:0.0:1789760715.651021:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534057856:12345-0@lo:400:kworker.0 Request processed in 1756us (2905us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760715.651026:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00100000:1.0:1789760715.651277:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760715.651688:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760715.651713:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c32ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534058240:0@lo:400:kworker.0 00000100:00100000:0.0:1789760715.651744:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760715.651760:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd99180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534058752:0@lo:400:kworker.0 00000100:00100000:0.0:1789760715.651779:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760715.651809:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534058368 00000100:00100000:0.0:1789760715.651813:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534058368 00000100:00100000:0.0:1789760715.651841:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534058880 00000100:00100000:0.0:1789760715.651844:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534058880 00000100:00100000:0.0:1789760715.651856:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534057984 00000100:00100000:0.0:1789760715.651859:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534057984 00000100:00100000:0.0:1789760715.651868:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534057728 00000100:00100000:0.0:1789760715.651871:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534057728 00000100:00100000:0.0:1789760715.651880:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534058240 00000100:00100000:0.0:1789760715.651882:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534058240 00000100:00100000:0.0:1789760715.651893:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534058752 00000100:00100000:0.0:1789760715.651895:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534058752 00000100:00100000:0.0:1789760715.651920:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 843 00000100:00100000:0.0:1789760715.651928:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3475:x1876699534058368:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760715.651981:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3475:x1876699534058368:12345-0@lo:400:kworker.0 Request processed in 53us (3897us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760715.651992:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 843 00000100:00100000:0.0:1789760715.652016:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 844 00000100:00100000:0.0:1789760715.652025:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3475:x1876699534058880:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.652277:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 845 00000100:00100000:2.0:1789760715.652308:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3473:x1876699534057984:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760715.652415:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3475:x1876699534058880:12345-0@lo:400:kworker.0 Request processed in 392us (4017us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760715.652422:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 844 00000100:00100000:0.0:1789760715.652442:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 846 00000100:00100000:0.0:1789760715.652449:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987257100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534057728:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.652470:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 847 00000100:00100000:1.0:1789760715.652479:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534058240:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.652545:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534058240:12345-0@lo:400:kworker.0 Request processed in 67us (800us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760715.652546:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987257100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534057728:12345-0@lo:400:kworker.0 Request processed in 96us (857us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760715.652552:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 847 00000100:00100000:2.0:1789760715.652590:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfef800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3473:x1876699534057984:12345-0@lo:400:kworker.0 Request processed in 305us (2494us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760715.652629:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 846 00000100:00100000:1.0:1789760715.652633:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 848 00000100:00100000:1.0:1789760715.652641:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534058752:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760715.652661:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534057600 00000100:00100000:0.0:1789760715.652666:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534057600 00000100:00100000:1.0:1789760715.652675:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534058752:12345-0@lo:400:kworker.0 Request processed in 33us (892us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760715.652682:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 848 00000100:00100000:0.0:1789760715.652697:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:1.0:1789760715.652702:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534058496 00000100:00100000:0.0:1789760715.652706:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04989032c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3473:x1876699534057600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.652706:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534058496 00000100:00100000:1.0:1789760715.652734:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534059008 00000100:00100000:1.0:1789760715.652738:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534059008 00000100:00100000:0.0:1789760715.652742:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04989032c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3473:x1876699534057600:12345-0@lo:400:kworker.0 Request processed in 38us (4715us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760715.652750:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:1.0:1789760715.652757:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789760715.652762:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534058496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760715.652796:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534058496:12345-0@lo:400:kworker.0 Request processed in 32us (2661us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760715.652803:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:2.0:1789760715.652806:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 845 00000100:00080000:2.0:1789760715.652822:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760715.652826:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760715.652828:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789760715.652854:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2dea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534059136:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760715.652938:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b645180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3473:1876699534057984:0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.652954:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd9b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534058496:0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.652972:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04884218380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534059008:0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.652984:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b647100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3473:1876699534057600:0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.653016:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd99c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3475:1876699534058368:0@lo:400:kworker.0 00000100:00100000:2.0:1789760715.653025:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04884219500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3475:1876699534058880:0@lo:400:kworker.0 00002000:00010000:0.0:1789760715.661671:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18841600 -> 18845696 00010000:00010000:0.0:1789760715.661690:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe544d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.661700:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe544d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.661706:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe544d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.661710:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760715.661713:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe544d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.661719:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe544d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.661735:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe544d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.661741:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe544d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18841600->18845695] (req 18841600->18845695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.661828:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508266880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36792us (37231us total) trans 4294971920 rc 0/0 00000100:00100000:0.0:1789760715.661838:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4632 00000100:00100000:0.0:1789760715.661848:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694689136144 time 694729729570 state 0x2 ref 0 00000100:00100000:2.0:1789760715.670573:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760715.703063:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18845696 -> 18849792 00010000:00010000:0.0:1789760715.703083:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe544de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.703095:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe544de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.703105:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe544de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.703113:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760715.703118:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe544de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.703127:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe544de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.703152:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe544de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.703161:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe544de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18845696->18849791] (req 18845696->18849791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.703256:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508267008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32367us (32687us total) trans 4294971921 rc 0/0 00000100:00100000:0.0:1789760715.703275:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4633 00000100:00100000:0.0:1789760715.703327:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694729989986 time 694775583275 state 0x2 ref 0 00000100:00100000:2.0:1789760715.713624:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760715.713720:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508267136 00000100:00100000:0.0:1789760715.713730:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508267136 00000100:00100000:0.0:1789760715.713784:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694729989986 time 694775583275 state 0x0 ref 1 00000100:00100000:0.0:1789760715.713800:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694775843691 time 694775583275 state 0x1 ref 1 00000100:00100000:0.0:1789760715.713880:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694775843691 time 694818590673 state 0x0 ref 1 00000100:00100000:0.0:1789760715.713893:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508267136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760715.713938:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18849792, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9007 00010000:00010000:0.0:1789760715.713946:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.713960:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760715.713965:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.713972:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.713986:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760715.748903:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18849792 -> 18853888 00010000:00010000:0.0:1789760715.748922:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.748936:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.748945:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.748953:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760715.748958:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.748967:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.748987:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.748995:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18849792->18853887] (req 18849792->18853887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.749093:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508267136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35196us (35474us total) trans 4294971922 rc 0/0 00000100:00100000:0.0:1789760715.749103:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4634 00000100:00100000:0.0:1789760715.749113:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694775843691 time 694818590673 state 0x2 ref 0 00000001:00080000:2.0:1789760715.749621:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971922 is committed 00000100:00100000:1.0:1789760715.757547:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760715.757792:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508267264 00000100:00100000:0.0:1789760715.757801:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508267264 00000100:00100000:0.0:1789760715.757853:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694775843691 time 694818590673 state 0x0 ref 1 00000100:00100000:0.0:1789760715.757869:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694818851089 time 694818590673 state 0x1 ref 1 00000100:00100000:0.0:1789760715.757932:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694818851089 time 694862642887 state 0x0 ref 1 00000100:00100000:0.0:1789760715.757945:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508267264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760715.757991:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18853888, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9007 00010000:00010000:0.0:1789760715.758000:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.758016:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760715.758020:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.758027:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.758047:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760715.814005:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971923 is committed 00002000:00010000:2.0:1789760715.814069:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18853888 -> 18857984 00010000:00010000:2.0:1789760715.814085:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.814108:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.814117:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.814124:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760715.814129:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.814138:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.814156:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.814163:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe544ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18853888->18857983] (req 18853888->18857983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760715.814770:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508267264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56820us (57272us total) trans 4294971923 rc 0/0 00000100:00100000:2.0:1789760715.814782:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4635 00000100:00100000:2.0:1789760715.814792:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694818851089 time 694862642887 state 0x2 ref 0 00000100:00100000:1.0:1789760715.825889:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760715.825975:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508267392 00000100:00100000:0.0:1789760715.825984:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508267392 00000100:00100000:0.0:1789760715.826040:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694818851089 time 694862642887 state 0x0 ref 1 00000100:00100000:0.0:1789760715.826055:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694862903303 time 694862642887 state 0x1 ref 1 00000100:00100000:0.0:1789760715.826083:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694862903303 time 694930794050 state 0x0 ref 1 00000100:00100000:0.0:1789760715.826107:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508267392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760715.826161:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18857984, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9007 00010000:00010000:0.0:1789760715.826171:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe544f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.826187:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760715.826191:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.826197:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe544f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18857984->18862079] (req 18857984->18862079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.826211:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe544f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18857984->18862079] (req 18857984->18862079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760715.869315:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971924 is committed 00000100:00100000:1.0:1789760715.873932:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760715.874059:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508267520 00000100:00100000:0.0:1789760715.874069:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508267520 00000100:00100000:0.0:1789760715.874125:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694862903303 time 694930794050 state 0x0 ref 1 00000100:00100000:0.0:1789760715.874140:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694931054466 time 694930794050 state 0x1 ref 1 00000100:00100000:0.0:1789760715.874257:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694931054466 time 694978965825 state 0x0 ref 1 00000100:00100000:0.0:1789760715.874282:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508267520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760715.874376:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18862080, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9007 00010000:00010000:0.0:1789760715.874385:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe544fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.874402:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789760715.874406:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760715.874413:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe544fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.874427:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe544fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760715.907514:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971925 is committed 00002000:00010000:1.0:1789760715.907833:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18862080 -> 18866176 00010000:00010000:1.0:1789760715.907848:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe544fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.907867:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe544fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.907877:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe544fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.907884:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760715.907888:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe544fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.907897:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe544fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.907920:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe544fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760715.907928:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe544fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18862080->18866175] (req 18862080->18866175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760715.908026:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508267520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33741us (34098us total) trans 4294971925 rc 0/0 00000100:00100000:1.0:1789760715.908036:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4637 00000100:00100000:1.0:1789760715.908046:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694931054466 time 694978965825 state 0x2 ref 0 00000100:00100000:1.0:1789760715.912958:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760715.913049:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508267648 00000100:00100000:2.0:1789760715.913059:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508267648 00000100:00100000:2.0:1789760715.913115:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694931054466 time 694978965825 state 0x0 ref 1 00000100:00100000:2.0:1789760715.913130:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694979226241 time 694978965825 state 0x1 ref 1 00000100:00100000:2.0:1789760715.913193:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694979226241 time 695017903035 state 0x0 ref 1 00000100:00100000:2.0:1789760715.913206:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508267648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760715.913256:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18866176, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9007 00010000:00010000:2.0:1789760715.913275:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54501 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.913422:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760715.913428:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760715.913435:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54501 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760715.913455:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54501 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760715.950720:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18866176 -> 18870272 00010000:00010000:0.0:1789760715.950737:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54501 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.950750:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54501 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.950759:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54501 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.950767:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760715.950772:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54501 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.950782:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54501 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.950802:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54501 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.950810:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54501 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18866176->18870271] (req 18866176->18870271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760715.950909:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508267648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37699us (37954us total) trans 4294971926 rc 0/0 00000100:00100000:0.0:1789760715.950921:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4638 00000100:00100000:0.0:1789760715.950930:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 694979226241 time 695017903035 state 0x2 ref 0 00000100:00100000:2.0:1789760715.955935:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760715.999132:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971927 is committed 00002000:00010000:0.0:1789760715.999238:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18870272 -> 18874368 00010000:00010000:0.0:1789760715.999255:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54508 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.999277:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54508 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.999311:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54508 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.999323:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760715.999327:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54508 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.999335:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54508 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.999424:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54508 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760715.999434:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54508 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18870272->18874367] (req 18870272->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.000141:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508267776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43661us (44209us total) trans 4294971927 rc 0/0 00000100:00100000:0.0:1789760716.000152:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4639 00000100:00100000:0.0:1789760716.000161:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695018163451 time 695061170312 state 0x2 ref 0 00000100:00100000:1.0:1789760716.010615:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760716.010734:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508267904 00000100:00100000:0.0:1789760716.010744:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508267904 00000100:00100000:0.0:1789760716.011058:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695018163451 time 695061170312 state 0x0 ref 1 00000100:00100000:0.0:1789760716.011074:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695061430728 time 695061170312 state 0x1 ref 1 00000100:00100000:0.0:1789760716.011193:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695061430728 time 695115901411 state 0x0 ref 1 00000100:00100000:0.0:1789760716.011211:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508267904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.011259:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18874368, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9007 00010000:00010000:0.0:1789760716.011279:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5450f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.011432:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.011438:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.011444:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5450f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.011460:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5450f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760716.053924:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18874368 -> 18878464 00002000:00010000:0.0:1789760716.053934:0:8108:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:0.0:1789760716.053949:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5450f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.053962:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5450f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.053971:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5450f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.053979:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760716.053984:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5450f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.053993:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5450f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.054013:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5450f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.054021:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5450f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18874368->18878463] (req 18874368->18878463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.054114:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508267904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42900us (43504us total) trans 4294971928 rc 0/0 00000100:00100000:0.0:1789760716.054125:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4640 00000100:00100000:0.0:1789760716.054135:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695061430728 time 695115901411 state 0x2 ref 0 00000100:00100000:2.0:1789760716.057552:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760716.057686:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508268032 00000100:00100000:0.0:1789760716.057695:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508268032 00000100:00100000:0.0:1789760716.057748:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695061430728 time 695115901411 state 0x0 ref 1 00000100:00100000:0.0:1789760716.057763:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695116161827 time 695115901411 state 0x1 ref 1 00000100:00100000:0.0:1789760716.057793:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695116161827 time 695162503723 state 0x0 ref 1 00000100:00100000:0.0:1789760716.057803:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508268032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.057851:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18878464, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9807 00010000:00010000:0.0:1789760716.057969:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54516 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.057986:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.057990:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.057996:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54516 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.058010:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54516 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.085105:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971929 is committed 00002000:00010000:0.0:1789760716.085340:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18878464 -> 18882560 00010000:00010000:0.0:1789760716.085355:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54516 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.085367:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54516 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.085376:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54516 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.085383:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760716.085388:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54516 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.085396:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54516 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.085421:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54516 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.085428:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54516 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18878464->18882559] (req 18878464->18882559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.085580:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508268032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27773us (28083us total) trans 4294971929 rc 0/0 00000100:00100000:0.0:1789760716.085590:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4641 00000100:00100000:0.0:1789760716.085599:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695116161827 time 695162503723 state 0x2 ref 0 00000100:00100000:2.0:1789760716.089958:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760716.090060:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508268160 00000100:00100000:0.0:1789760716.090070:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508268160 00000100:00100000:0.0:1789760716.090126:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695116161827 time 695162503723 state 0x0 ref 1 00000100:00100000:0.0:1789760716.090142:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695162764139 time 695162503723 state 0x1 ref 1 00000100:00100000:0.0:1789760716.090221:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695162764139 time 695194930820 state 0x0 ref 1 00000100:00100000:0.0:1789760716.090234:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508268160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.090315:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18882560, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9807 00010000:00010000:0.0:1789760716.090325:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5451d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.090340:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.090345:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.090351:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5451d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.090366:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5451d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18882560->18886655] (req 18882560->18886655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.127926:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971930 is committed 00000100:00100000:2.0:1789760716.134055:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760716.134173:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508268288 00000100:00100000:0.0:1789760716.134190:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508268288 00000100:00100000:0.0:1789760716.134244:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695162764139 time 695194930820 state 0x0 ref 1 00000100:00100000:0.0:1789760716.134273:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695195191236 time 695194930820 state 0x1 ref 1 00000100:00100000:0.0:1789760716.134350:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695195191236 time 695239060053 state 0x0 ref 1 00000100:00100000:0.0:1789760716.134363:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508268288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.134681:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18886656, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9807 00010000:00010000:0.0:1789760716.134692:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54524 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.134707:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.134712:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.134718:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54524 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.134732:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54524 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760716.181403:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18886656 -> 18890752 00010000:00010000:0.0:1789760716.181419:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54524 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.181432:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54524 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.181441:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54524 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.181449:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760716.181453:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54524 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.181463:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54524 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.181483:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54524 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.181491:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54524 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18886656->18890751] (req 18886656->18890751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.182235:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508268288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47219us (47533us total) trans 4294971931 rc 0/0 00000100:00100000:0.0:1789760716.182252:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4643 00000100:00100000:0.0:1789760716.182278:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695195191236 time 695239060053 state 0x2 ref 0 00000100:00100000:2.0:1789760716.186462:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760716.186538:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508268416 00000100:00100000:0.0:1789760716.186547:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508268416 00000100:00100000:0.0:1789760716.186609:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695195191236 time 695239060053 state 0x0 ref 1 00000100:00100000:0.0:1789760716.186633:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695239320469 time 695239060053 state 0x1 ref 1 00000100:00100000:0.0:1789760716.186657:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695239320469 time 695291368350 state 0x0 ref 1 00000100:00100000:0.0:1789760716.186668:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508268416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.186712:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18890752, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9807 00010000:00010000:0.0:1789760716.186721:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5452b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.186736:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.186740:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.186746:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5452b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18890752->18894847] (req 18890752->18894847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.186758:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5452b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18890752->18894847] (req 18890752->18894847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.222020:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760716.253388:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18894848 -> 18898944 00010000:00010000:0.0:1789760716.253405:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54532 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.253418:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54532 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.253427:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54532 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.253435:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760716.253439:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54532 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.253447:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54532 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.253468:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54532 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.253475:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54532 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18894848->18898943] (req 18894848->18898943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.253682:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d45050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508268544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31464us (31667us total) trans 4294971933 rc 0/0 00000100:00100000:0.0:1789760716.253693:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4645 00000100:00100000:0.0:1789760716.253703:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695291628766 time 695326915302 state 0x2 ref 0 00000100:00100000:0.0:1789760716.258053:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760716.258209:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508268672 00000100:00100000:1.0:1789760716.258219:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508268672 00000100:00100000:1.0:1789760716.258283:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695291628766 time 695326915302 state 0x0 ref 1 00000100:00100000:1.0:1789760716.258519:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695327175718 time 695326915302 state 0x1 ref 1 00000100:00100000:1.0:1789760716.258577:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695327175718 time 695363287037 state 0x0 ref 1 00000100:00100000:1.0:1789760716.258589:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508268672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760716.258643:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18898944, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9807 00010000:00010000:1.0:1789760716.258652:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54539 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.258667:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760716.258672:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.258678:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54539 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.258691:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54539 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760716.307423:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18898944 -> 18903040 00010000:00010000:1.0:1789760716.307443:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54539 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.307461:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54539 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.307469:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54539 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.307476:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760716.307481:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54539 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.307489:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54539 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.307509:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54539 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.307516:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54539 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18898944->18903039] (req 18898944->18903039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.307673:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508268672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49020us (49562us total) trans 4294971934 rc 0/0 00000100:00100000:1.0:1789760716.307684:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4646 00000100:00100000:1.0:1789760716.307693:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695327175718 time 695363287037 state 0x2 ref 0 00000100:00100000:2.0:1789760716.310144:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760716.310679:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695363547453 time 695415387487 state 0x0 ref 1 00000100:00100000:1.0:1789760716.310693:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508268800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760716.310740:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18903040, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9807 00010000:00010000:1.0:1789760716.310749:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54540 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.310764:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760716.310769:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.310775:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54540 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.310789:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54540 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760716.351474:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18903040 -> 18907136 00010000:00010000:0.0:1789760716.351490:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54540 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.351502:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54540 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.351511:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54540 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.351518:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760716.351523:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54540 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.351532:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54540 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.351550:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54540 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.351569:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54540 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18903040->18907135] (req 18903040->18907135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.351670:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+6:2409:x1876699508268800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40975us (41527us total) trans 4294971935 rc 0/0 00000100:00100000:0.0:1789760716.351681:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4647 00000100:00100000:0.0:1789760716.351691:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695363547453 time 695415387487 state 0x2 ref 0 00000100:00100000:2.0:1789760716.355807:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760716.355885:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508268928 00000100:00100000:0.0:1789760716.355894:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508268928 00000100:00100000:0.0:1789760716.355934:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695363547453 time 695415387487 state 0x0 ref 1 00000100:00100000:0.0:1789760716.355946:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695415647903 time 695415387487 state 0x1 ref 1 00000100:00100000:0.0:1789760716.355971:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695415647903 time 695460682720 state 0x0 ref 1 00000100:00100000:0.0:1789760716.355979:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508268928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.356018:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18907136, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9807 00010000:00010000:0.0:1789760716.356024:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54547 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.356037:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.356040:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.356044:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54547 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.356054:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54547 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760716.382021:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18907136 -> 18911232 00010000:00010000:2.0:1789760716.382036:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54547 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.382048:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54547 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.382056:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54547 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.382063:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760716.382068:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54547 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.382083:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54547 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.382103:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54547 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.382111:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54547 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18907136->18911231] (req 18907136->18911231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.382278:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508268928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26293us (26474us total) trans 4294971936 rc 0/0 00000100:00100000:2.0:1789760716.382322:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4648 00000100:00100000:2.0:1789760716.382332:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695415647903 time 695460682720 state 0x2 ref 0 00000100:00100000:2.0:1789760716.387165:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760716.387315:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508269056 00000100:00100000:1.0:1789760716.387324:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508269056 00000100:00100000:1.0:1789760716.387372:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695415647903 time 695460682720 state 0x0 ref 1 00000100:00100000:1.0:1789760716.387387:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695460943136 time 695460682720 state 0x1 ref 1 00000100:00100000:1.0:1789760716.387415:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695460943136 time 695492125606 state 0x0 ref 1 00000100:00100000:1.0:1789760716.387427:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508269056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760716.387472:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18911232, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9807 00010000:00010000:1.0:1789760716.387480:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5454e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.387501:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760716.387506:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.387512:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5454e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.387525:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5454e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760716.468278:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18911232 -> 18915328 00010000:00010000:0.0:1789760716.468323:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5454e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.468335:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5454e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.468344:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5454e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.468351:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760716.468356:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5454e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.468366:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5454e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.468386:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5454e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.468394:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5454e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18911232->18915327] (req 18911232->18915327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.468518:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508269056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 81087us (81357us total) trans 4294971937 rc 0/0 00000100:00100000:0.0:1789760716.468528:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4649 00000100:00100000:0.0:1789760716.468537:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695460943136 time 695492125606 state 0x2 ref 0 00000100:00100000:0.0:1789760716.488540:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508269184 00000100:00100000:0.0:1789760716.488550:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508269184 00000100:00100000:0.0:1789760716.488612:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695460943136 time 695492125606 state 0x0 ref 1 00000100:00100000:0.0:1789760716.488628:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695492386022 time 695492125606 state 0x1 ref 1 00000100:00100000:0.0:1789760716.488660:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695492386022 time 695593371128 state 0x0 ref 1 00000100:00100000:0.0:1789760716.488673:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508269184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.488720:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18915328, mtime: 0x6aad94cc, atime: 0x0, ctime: 0x6aad94cc, blocks: 0x9807 00010000:00010000:0.0:1789760716.488729:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54555 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.488744:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.488748:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.488754:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54555 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.488768:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54555 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760716.549008:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18915328 -> 18919424 00002000:00010000:2.0:1789760716.549018:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760716 -> 1789760717 00002000:00010000:2.0:1789760716.549021:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760716 -> 1789760717 00010000:00010000:2.0:1789760716.549035:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54555 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.549049:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54555 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.549058:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54555 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.549066:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760716.549071:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54555 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.549081:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54555 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.549101:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54555 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.549110:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54555 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18915328->18919423] (req 18915328->18919423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.549217:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508269184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60541us (60813us total) trans 4294971938 rc 0/0 00000100:00100000:2.0:1789760716.549229:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4650 00000100:00100000:2.0:1789760716.549238:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695492386022 time 695593371128 state 0x2 ref 0 00000100:00100000:2.0:1789760716.558347:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760716.558480:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508269312 00000100:00100000:0.0:1789760716.558490:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508269312 00000100:00100000:0.0:1789760716.558544:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695492386022 time 695593371128 state 0x0 ref 1 00000100:00100000:0.0:1789760716.558572:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695593631544 time 695593371128 state 0x1 ref 1 00000100:00100000:0.0:1789760716.558600:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695593631544 time 695663311202 state 0x0 ref 1 00000100:00100000:0.0:1789760716.558621:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508269312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.558692:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18919424, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:0.0:1789760716.558702:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5455c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.558718:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.558723:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.558730:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5455c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.558744:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5455c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760716.589567:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18919424 -> 18923520 00010000:00010000:2.0:1789760716.589583:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5455c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.589596:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5455c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.589674:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5455c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.589682:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760716.589688:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5455c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.589697:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5455c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.589719:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5455c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.589727:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5455c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18919424->18923519] (req 18919424->18923519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.589825:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508269312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31208us (31483us total) trans 4294971939 rc 0/0 00000100:00100000:2.0:1789760716.589836:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4651 00000100:00100000:2.0:1789760716.589846:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695593631544 time 695663311202 state 0x2 ref 0 00000100:00100000:1.0:1789760716.593134:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508269440 00000100:00100000:1.0:1789760716.593144:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508269440 00000100:00100000:1.0:1789760716.593202:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695593631544 time 695663311202 state 0x0 ref 1 00000100:00100000:1.0:1789760716.593218:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695663571618 time 695663311202 state 0x1 ref 1 00000100:00100000:1.0:1789760716.593246:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695663571618 time 695697956192 state 0x0 ref 1 00000100:00100000:1.0:1789760716.593267:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508269440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760716.593344:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18923520, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:1.0:1789760716.593353:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54563 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.593369:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760716.593373:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.593379:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54563 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.593399:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54563 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760716.623924:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18923520 -> 18927616 00010000:00010000:2.0:1789760716.623939:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54563 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.623958:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54563 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.623967:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54563 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.623975:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760716.623980:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54563 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.623990:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54563 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.624010:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54563 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.624018:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54563 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18923520->18927615] (req 18923520->18927615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760716.624112:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfeea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508269440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30850us (31190us total) trans 4294971940 rc 0/0 00000100:00100000:2.0:1789760716.624123:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4652 00000100:00100000:2.0:1789760716.624133:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695663571618 time 695697956192 state 0x2 ref 0 00000100:00100000:2.0:1789760716.629006:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508269568 00000100:00100000:2.0:1789760716.629017:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508269568 00000100:00100000:2.0:1789760716.629072:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695663571618 time 695697956192 state 0x0 ref 1 00000100:00100000:2.0:1789760716.629089:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695698216608 time 695697956192 state 0x1 ref 1 00000100:00100000:2.0:1789760716.629124:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695698216608 time 695733834281 state 0x0 ref 1 00000100:00100000:2.0:1789760716.629137:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508269568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760716.629256:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18927616, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:2.0:1789760716.629277:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5456a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.629336:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789760716.629340:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760716.629347:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5456a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760716.629363:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5456a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760716.658321:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18927616 -> 18931712 00010000:00010000:0.0:1789760716.658340:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5456a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.658354:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5456a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.658362:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5456a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.658369:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760716.658374:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5456a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.658383:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5456a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.658408:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5456a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.658416:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5456a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18927616->18931711] (req 18927616->18931711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.658507:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508269568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29367us (29725us total) trans 4294971941 rc 0/0 00000100:00100000:0.0:1789760716.658519:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4653 00000100:00100000:0.0:1789760716.658529:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695698216608 time 695733834281 state 0x2 ref 0 00000100:00100000:2.0:1789760716.667815:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760716.668099:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508269696 00000100:00100000:0.0:1789760716.668109:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508269696 00000100:00100000:0.0:1789760716.668165:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695698216608 time 695733834281 state 0x0 ref 1 00000100:00100000:0.0:1789760716.668179:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695734094697 time 695733834281 state 0x1 ref 1 00000100:00100000:0.0:1789760716.668210:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695734094697 time 695772920673 state 0x0 ref 1 00000100:00100000:0.0:1789760716.668223:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508269696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.668279:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18931712, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:0.0:1789760716.668327:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54571 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.668344:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.668349:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.668355:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54571 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.668368:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54571 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.671191:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499bfed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534059520:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789760716.671269:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760716.671485:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534059520 00000100:00100000:2.0:1789760716.671494:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534059520 00000100:00100000:2.0:1789760716.671545:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:2.0:1789760716.671555:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534059520:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789760716.671591:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789760716.674116:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534059520:12345-0@lo:41:osp-pre-0-1.0 Request processed in 2552us (2844us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760716.674131:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:0.0:1789760716.674224:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499bfed880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534059520:0@lo:41:osp-pre-0-1.0 00000001:00080000:2.0:1789760716.691125:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971942 is committed 00002000:00010000:0.0:1789760716.691427:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18931712 -> 18935808 00010000:00010000:0.0:1789760716.691441:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54571 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.691454:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54571 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.691463:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54571 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.691471:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760716.691476:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54571 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.691485:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54571 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.691505:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54571 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.691514:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54571 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18931712->18935807] (req 18931712->18935807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.691633:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508269696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23408us (23824us total) trans 4294971942 rc 0/0 00000100:00100000:0.0:1789760716.691640:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4654 00000100:00100000:0.0:1789760716.691647:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695734094697 time 695772920673 state 0x2 ref 0 00000100:00100000:2.0:1789760716.695879:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760716.695996:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508269824 00000100:00100000:1.0:1789760716.696006:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508269824 00000100:00100000:1.0:1789760716.696062:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695734094697 time 695772920673 state 0x0 ref 1 00000100:00100000:1.0:1789760716.696076:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695773181089 time 695772920673 state 0x1 ref 1 00000100:00100000:1.0:1789760716.696138:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695773181089 time 695800848017 state 0x0 ref 1 00000100:00100000:1.0:1789760716.696149:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508269824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760716.696205:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18935808, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:1.0:1789760716.696215:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54578 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.696232:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760716.696236:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.696243:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54578 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.696267:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54578 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18935808->18939903] (req 18935808->18939903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.734030:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971943 is committed 00000100:00100000:0.0:1789760716.747343:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508269952 00000100:00100000:0.0:1789760716.747357:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508269952 00000100:00100000:0.0:1789760716.747420:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695773181089 time 695800848017 state 0x0 ref 1 00000100:00100000:0.0:1789760716.747436:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695801108433 time 695800848017 state 0x1 ref 1 00000100:00100000:0.0:1789760716.747472:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695801108433 time 695852182426 state 0x0 ref 1 00000100:00100000:0.0:1789760716.747486:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508269952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.747535:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18939904, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:0.0:1789760716.747544:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5457f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.747560:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.747564:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.747571:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5457f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.747589:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5457f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760716.773664:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18939904 -> 18944000 00010000:00010000:1.0:1789760716.773681:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5457f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.773694:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5457f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.773703:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5457f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.773710:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760716.773715:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5457f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.773724:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5457f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.773743:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5457f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.773751:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5457f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18939904->18943999] (req 18939904->18943999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.773837:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508269952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26349us (28842us total) trans 4294971944 rc 0/0 00000100:00100000:1.0:1789760716.773849:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4656 00000100:00100000:1.0:1789760716.773859:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695801108433 time 695852182426 state 0x2 ref 0 00000100:00100000:2.0:1789760716.776702:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760716.776820:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508270080 00000100:00100000:0.0:1789760716.776832:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508270080 00000100:00100000:0.0:1789760716.776891:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695801108433 time 695852182426 state 0x0 ref 1 00000100:00100000:0.0:1789760716.776908:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695852442842 time 695852182426 state 0x1 ref 1 00000100:00100000:0.0:1789760716.776978:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695852442842 time 695881687253 state 0x0 ref 1 00000100:00100000:0.0:1789760716.776991:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508270080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.777040:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18944000, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:0.0:1789760716.777065:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54586 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.777083:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789760716.777088:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.777094:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54586 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.777108:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54586 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18944000->18948095] (req 18944000->18948095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.797843:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971945 is committed 00000100:00100000:2.0:1789760716.801731:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760716.823185:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971946 is committed 00002000:00010000:1.0:1789760716.823281:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18948096 -> 18952192 00010000:00010000:1.0:1789760716.823329:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5458d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.823342:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5458d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.823351:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5458d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.823359:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760716.823364:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5458d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.823373:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5458d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.823393:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5458d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.823401:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5458d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18948096->18952191] (req 18948096->18952191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.823554:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508270208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 21502us (21826us total) trans 4294971946 rc 0/0 00000100:00100000:1.0:1789760716.823565:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4658 00000100:00100000:1.0:1789760716.823574:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695881947669 time 695906744701 state 0x2 ref 0 00000100:00100000:1.0:1789760716.827672:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508270336 00000100:00100000:1.0:1789760716.827683:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508270336 00000100:00100000:1.0:1789760716.827740:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695881947669 time 695906744701 state 0x0 ref 1 00000100:00100000:1.0:1789760716.827754:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695907005117 time 695906744701 state 0x1 ref 1 00000100:00100000:1.0:1789760716.827793:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695907005117 time 695932503632 state 0x0 ref 1 00000100:00100000:1.0:1789760716.827805:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508270336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760716.827860:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18952192, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:1.0:1789760716.827869:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54594 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.827884:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760716.827889:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.827895:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54594 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.827909:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54594 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.850535:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971947 is committed 00002000:00010000:1.0:1789760716.850579:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18952192 -> 18956288 00010000:00010000:1.0:1789760716.850596:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54594 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.850608:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54594 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.850617:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54594 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.850637:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760716.850642:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54594 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.850652:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54594 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.850671:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54594 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.850679:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54594 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18952192->18956287] (req 18952192->18956287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.850779:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508270336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22970us (23242us total) trans 4294971947 rc 0/0 00000100:00100000:1.0:1789760716.850793:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4659 00000100:00100000:1.0:1789760716.850802:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695907005117 time 695932503632 state 0x2 ref 0 00000100:00100000:2.0:1789760716.853882:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760716.879266:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971948 is committed 00002000:00010000:0.0:1789760716.879466:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18956288 -> 18960384 00010000:00010000:0.0:1789760716.879481:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5459b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.879493:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5459b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.879502:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5459b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.879509:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760716.879513:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5459b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.879523:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5459b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.879540:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5459b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.879547:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5459b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18956288->18960383] (req 18956288->18960383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.879637:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508270464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25566us (25760us total) trans 4294971948 rc 0/0 00000100:00100000:0.0:1789760716.879647:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4660 00000100:00100000:0.0:1789760716.879657:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695932764048 time 695958766561 state 0x2 ref 0 00000100:00100000:2.0:1789760716.883483:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760716.883616:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508270592 00000100:00100000:1.0:1789760716.883625:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508270592 00000100:00100000:1.0:1789760716.883670:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695932764048 time 695958766561 state 0x0 ref 1 00000100:00100000:1.0:1789760716.883684:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695959026977 time 695958766561 state 0x1 ref 1 00000100:00100000:1.0:1789760716.883740:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695959026977 time 695988450866 state 0x0 ref 1 00000100:00100000:1.0:1789760716.883752:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508270592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760716.883793:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18960384, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:1.0:1789760716.883802:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe545a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.883816:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=695s v=(0 0) 00010000:00010000:1.0:1789760716.883820:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.883827:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe545a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.883841:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe545a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760716.896740:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971949 is committed 00002000:00010000:1.0:1789760716.896800:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18960384 -> 18964480 00010000:00010000:1.0:1789760716.896816:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe545a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.896829:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe545a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.896839:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe545a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.896847:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760716.896852:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe545a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.896861:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe545a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.896880:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe545a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.896888:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe545a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18960384->18964479] (req 18960384->18964479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760716.896971:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508270592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13215us (13492us total) trans 4294971949 rc 0/0 00000100:00100000:1.0:1789760716.896981:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4661 00000100:00100000:1.0:1789760716.896990:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695959026977 time 695988450866 state 0x2 ref 0 00000100:00100000:1.0:1789760716.899848:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760716.899937:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508270720 00000100:00100000:0.0:1789760716.899948:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508270720 00000100:00100000:0.0:1789760716.900000:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695959026977 time 695988450866 state 0x0 ref 1 00000100:00100000:0.0:1789760716.900016:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695988711282 time 695988450866 state 0x1 ref 1 00000100:00100000:0.0:1789760716.900047:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695988711282 time 696004757921 state 0x0 ref 1 00000100:00100000:0.0:1789760716.900069:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508270720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.900322:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18964480, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:0.0:1789760716.900333:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.900347:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760716.900352:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.900358:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.900371:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760716.916852:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18964480 -> 18968576 00010000:00010000:0.0:1789760716.916867:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.916881:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.916894:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.916902:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760716.916907:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.916917:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.916936:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.916944:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18964480->18968575] (req 18964480->18968575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.917029:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508270720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16958us (17184us total) trans 4294971950 rc 0/0 00000100:00100000:0.0:1789760716.917038:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4662 00000100:00100000:0.0:1789760716.917047:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695988711282 time 696004757921 state 0x2 ref 0 00000100:00100000:2.0:1789760716.922630:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760716.923080:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508270848 00000100:00100000:0.0:1789760716.923091:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508270848 00000100:00100000:0.0:1789760716.923144:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 695988711282 time 696004757921 state 0x0 ref 1 00000100:00100000:0.0:1789760716.923158:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696005018337 time 696004757921 state 0x1 ref 1 00000100:00100000:0.0:1789760716.923202:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696005018337 time 696027905746 state 0x0 ref 1 00000100:00100000:0.0:1789760716.923214:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508270848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760716.923270:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18968576, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:0.0:1789760716.923278:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.923328:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760716.923332:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760716.923339:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.923353:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760716.970421:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971951 is committed 00002000:00010000:0.0:1789760716.971016:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18968576 -> 18972672 00010000:00010000:0.0:1789760716.971033:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.971048:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.971059:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.971067:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760716.971073:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.971082:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.971105:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760716.971114:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18968576->18972671] (req 18968576->18972671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760716.971212:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508270848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47994us (48586us total) trans 4294971951 rc 0/0 00000100:00100000:0.0:1789760716.971222:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4663 00000100:00100000:0.0:1789760716.971233:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696005018337 time 696027905746 state 0x2 ref 0 00000100:00100000:1.0:1789760716.977279:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508270976 00000100:00100000:1.0:1789760716.977326:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508270976 00000100:00100000:1.0:1789760716.977383:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696005018337 time 696027905746 state 0x0 ref 1 00000100:00100000:1.0:1789760716.977401:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696028166162 time 696027905746 state 0x1 ref 1 00000100:00100000:1.0:1789760716.977477:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696028166162 time 696082186270 state 0x0 ref 1 00000100:00100000:1.0:1789760716.977490:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfeed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508270976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760716.977535:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18972672, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:1.0:1789760716.977545:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe545b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.977560:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760716.977565:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760716.977571:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe545b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760716.977585:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe545b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18972672->18976767] (req 18972672->18976767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.005047:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971952 is committed 00000100:00100000:1.0:1789760717.012881:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760717.012988:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508271104 00000100:00100000:0.0:1789760717.012999:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508271104 00000100:00100000:0.0:1789760717.013072:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696028166162 time 696082186270 state 0x0 ref 1 00000100:00100000:0.0:1789760717.013088:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696082446686 time 696082186270 state 0x1 ref 1 00000100:00100000:0.0:1789760717.013119:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696082446686 time 696117829544 state 0x0 ref 1 00000100:00100000:0.0:1789760717.013132:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508271104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760717.013182:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18976768, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00000100:00080000:2.0:1789760717.013213:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789760717.013366:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.013381:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.013386:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.013392:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.013407:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760717.038765:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18976768 -> 18980864 00010000:00010000:1.0:1789760717.038784:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.038802:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.038810:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.038817:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760717.038821:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.038830:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.038852:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.038860:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe545be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18976768->18980863] (req 18976768->18980863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.038963:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508271104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25829us (26086us total) trans 4294971953 rc 0/0 00000100:00100000:1.0:1789760717.038975:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4665 00000100:00100000:1.0:1789760717.038985:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696082446686 time 696117829544 state 0x2 ref 0 00000100:00100000:0.0:1789760717.042720:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:1.0:1789760717.065738:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18980864 -> 18984960 00010000:00010000:1.0:1789760717.065753:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe545c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.065764:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe545c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.065770:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe545c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.065775:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760717.065778:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe545c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.065785:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe545c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.065799:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe545c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.065805:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe545c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18980864->18984959] (req 18980864->18984959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.065883:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508271232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22975us (23167us total) trans 4294971954 rc 0/0 00000100:00100000:1.0:1789760717.065893:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4666 00000100:00100000:1.0:1789760717.065902:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696118089960 time 696147602606 state 0x2 ref 0 00000100:00100000:0.0:1789760717.073088:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:1.0:1789760717.096318:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18984960 -> 18989056 00010000:00010000:1.0:1789760717.096338:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe545cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.096353:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe545cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.096363:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe545cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.096370:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760717.096375:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe545cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.096386:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe545cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.096405:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe545cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.096414:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe545cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18984960->18989055] (req 18984960->18989055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.096491:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987257800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508271360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22553us (23409us total) trans 4294971955 rc 0/0 00000100:00100000:1.0:1789760717.096502:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4667 00000100:00100000:1.0:1789760717.096512:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696147863022 time 696178633876 state 0x2 ref 0 00000100:00100000:1.0:1789760717.099421:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508271488 00000100:00100000:1.0:1789760717.099432:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508271488 00000100:00100000:1.0:1789760717.099487:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696147863022 time 696178633876 state 0x0 ref 1 00000100:00100000:1.0:1789760717.099502:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696178894292 time 696178633876 state 0x1 ref 1 00000100:00100000:1.0:1789760717.099534:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696178894292 time 696204244205 state 0x0 ref 1 00000100:00100000:1.0:1789760717.099546:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508271488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760717.099596:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18989056, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:1.0:1789760717.099604:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.099630:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760717.099634:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.099640:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.099654:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.122354:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971956 is committed 00002000:00010000:1.0:1789760717.122532:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 18989056 -> 18993152 00010000:00010000:1.0:1789760717.122547:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.122565:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.122574:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.122581:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760717.122586:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.122595:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.122729:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.122738:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18989056->18993151] (req 18989056->18993151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.122842:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508271488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23292us (23493us total) trans 4294971956 rc 0/0 00000100:00100000:1.0:1789760717.122852:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4668 00000100:00100000:1.0:1789760717.122861:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696178894292 time 696204244205 state 0x2 ref 0 00000100:00100000:0.0:1789760717.129703:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760717.129901:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508271616 00000100:00100000:1.0:1789760717.129912:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508271616 00000100:00100000:1.0:1789760717.129970:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696178894292 time 696204244205 state 0x0 ref 1 00000100:00100000:1.0:1789760717.129984:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696204504621 time 696204244205 state 0x1 ref 1 00000100:00100000:1.0:1789760717.130017:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696204504621 time 696234727486 state 0x0 ref 1 00000100:00100000:1.0:1789760717.130029:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508271616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760717.130077:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18993152, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:1.0:1789760717.130086:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.130100:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760717.130103:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.130110:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18993152->18997247] (req 18993152->18997247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.130123:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe545da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18993152->18997247] (req 18993152->18997247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.176265:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971957 is committed 00000100:00100000:0.0:1789760717.184093:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760717.185351:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508271744 00000100:00100000:1.0:1789760717.185362:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508271744 00000100:00100000:1.0:1789760717.185420:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696204504621 time 696234727486 state 0x0 ref 1 00000100:00100000:1.0:1789760717.185434:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696234987902 time 696234727486 state 0x1 ref 1 00000100:00100000:1.0:1789760717.185466:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696234987902 time 696290176891 state 0x0 ref 1 00000100:00100000:1.0:1789760717.185483:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987257b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508271744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760717.185534:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 18997248, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:1.0:1789760717.185542:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.185557:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760717.185561:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.185567:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.186033:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [18997248->19001343] (req 18997248->19001343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.229433:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971958 is committed 00000100:00100000:1.0:1789760717.232836:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760717.232947:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508271872 00000100:00100000:0.0:1789760717.232957:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508271872 00000100:00100000:0.0:1789760717.233005:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696234987902 time 696290176891 state 0x0 ref 1 00000100:00100000:0.0:1789760717.233020:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696290437307 time 696290176891 state 0x1 ref 1 00000100:00100000:0.0:1789760717.233049:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696290437307 time 696337759985 state 0x0 ref 1 00000100:00100000:0.0:1789760717.233061:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508271872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760717.233107:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19001344, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:0.0:1789760717.233115:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe545e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.233129:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.233134:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.233140:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe545e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.233155:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe545e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760717.267338:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971959 is committed 00002000:00010000:1.0:1789760717.267488:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19001344 -> 19005440 00010000:00010000:1.0:1789760717.267504:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe545e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.267516:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe545e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.267524:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe545e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.267531:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760717.267537:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe545e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.267546:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe545e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.267574:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe545e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.267589:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe545e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19001344->19005439] (req 19001344->19005439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760717.267714:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508271872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34649us (34881us total) trans 4294971959 rc 0/0 00000100:00100000:1.0:1789760717.267724:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4671 00000100:00100000:1.0:1789760717.267744:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696290437307 time 696337759985 state 0x2 ref 0 00000100:00100000:0.0:1789760717.275404:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760717.321578:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971960 is committed 00000100:00100000:1.0:1789760717.337010:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760717.442057:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971961 is committed 00002000:00010000:2.0:1789760717.442168:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19009536 -> 19013632 00010000:00010000:2.0:1789760717.442356:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe545f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.442370:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe545f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.442379:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe545f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.442386:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760717.442391:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe545f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.442407:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe545f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.442429:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe545f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.442437:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe545f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19009536->19013631] (req 19009536->19013631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.442577:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508272128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 101153us (105575us total) trans 4294971961 rc 0/0 00000100:00100000:2.0:1789760717.442588:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4673 00000100:00100000:2.0:1789760717.442598:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696380591929 time 696446119358 state 0x2 ref 0 00000100:00100000:0.0:1789760717.470881:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760717.471020:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508272256 00000100:00100000:1.0:1789760717.471038:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508272256 00000100:00100000:1.0:1789760717.471091:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696380591929 time 696446119358 state 0x0 ref 1 00000100:00100000:1.0:1789760717.471104:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696446379774 time 696446119358 state 0x1 ref 1 00000100:00100000:1.0:1789760717.471134:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696446379774 time 696575844559 state 0x0 ref 1 00000100:00100000:1.0:1789760717.471147:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508272256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760717.471198:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19013632, mtime: 0x6aad94cd, atime: 0x0, ctime: 0x6aad94cd, blocks: 0x9807 00010000:00010000:1.0:1789760717.471208:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.471222:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:1.0:1789760717.471226:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760717.471232:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760717.471246:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760717.556660:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19013632 -> 19017728 00002000:00010000:2.0:1789760717.556666:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760717 -> 1789760718 00002000:00010000:2.0:1789760717.556669:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760717 -> 1789760718 00010000:00010000:2.0:1789760717.556682:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.556695:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.556705:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.556718:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760717.556722:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.556732:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.556752:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.556760:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe545fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19013632->19017727] (req 19013632->19017727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.556851:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508272256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 85701us (85974us total) trans 4294971962 rc 0/0 00000100:00100000:2.0:1789760717.556862:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4674 00000100:00100000:2.0:1789760717.556873:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696446379774 time 696575844559 state 0x2 ref 0 00000100:00100000:1.0:1789760717.565376:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00010000:00010000:2.0:1789760717.566484:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760717.632125:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971963 is committed 00000100:00100000:2.0:1789760717.641045:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508272512 00000100:00100000:2.0:1789760717.641054:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508272512 00000100:00100000:2.0:1789760717.641103:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696576104975 time 696670286784 state 0x0 ref 1 00000100:00100000:2.0:1789760717.641117:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696670547200 time 696670286784 state 0x1 ref 1 00000100:00100000:2.0:1789760717.641146:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696670547200 time 696745857531 state 0x0 ref 1 00000100:00100000:2.0:1789760717.641158:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508272512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760717.641208:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19021824, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:2.0:1789760717.641217:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5460b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.641232:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:2.0:1789760717.641236:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.641242:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5460b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.641265:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5460b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.676944:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971964 is committed 00002000:00010000:0.0:1789760717.677019:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19021824 -> 19025920 00010000:00010000:0.0:1789760717.677033:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5460b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.677046:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5460b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.677055:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5460b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.677068:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760717.677072:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5460b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.677080:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5460b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.677100:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5460b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.677106:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5460b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19021824->19025919] (req 19021824->19025919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.677457:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfefb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508272512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36294us (36550us total) trans 4294971964 rc 0/0 00000100:00100000:0.0:1789760717.677467:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4676 00000100:00100000:0.0:1789760717.677476:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696670547200 time 696745857531 state 0x2 ref 0 00000100:00100000:0.0:1789760717.683050:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760717.683129:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508272640 00000100:00100000:2.0:1789760717.683138:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508272640 00000100:00100000:2.0:1789760717.683236:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696670547200 time 696745857531 state 0x0 ref 1 00000100:00100000:2.0:1789760717.683259:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696746117947 time 696745857531 state 0x1 ref 1 00000100:00100000:2.0:1789760717.683433:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696746117947 time 696788142936 state 0x0 ref 1 00000100:00100000:2.0:1789760717.683445:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508272640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760717.683488:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19025920, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:2.0:1789760717.683497:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54612 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.683518:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:2.0:1789760717.683522:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.683528:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54612 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.683550:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54612 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.720677:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971965 is committed 00002000:00010000:2.0:1789760717.720964:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19025920 -> 19030016 00010000:00010000:2.0:1789760717.720981:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54612 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.720994:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54612 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.721003:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54612 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.721011:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760717.721016:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54612 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.721026:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54612 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.721047:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54612 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.721056:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54612 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19025920->19030015] (req 19025920->19030015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.721144:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508272640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37695us (38099us total) trans 4294971965 rc 0/0 00000100:00100000:2.0:1789760717.721156:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4677 00000100:00100000:2.0:1789760717.721178:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696746117947 time 696788142936 state 0x2 ref 0 00000100:00100000:1.0:1789760717.727923:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760717.728379:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508272768 00000100:00100000:0.0:1789760717.728389:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508272768 00000100:00100000:0.0:1789760717.728443:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696746117947 time 696788142936 state 0x0 ref 1 00000100:00100000:0.0:1789760717.728459:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696788403352 time 696788142936 state 0x1 ref 1 00000100:00100000:0.0:1789760717.728828:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696788403352 time 696833536362 state 0x0 ref 1 00000100:00100000:0.0:1789760717.728844:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508272768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760717.728893:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19030016, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:0.0:1789760717.728901:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54619 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.728916:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.728920:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.728925:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54619 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.728938:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54619 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19030016->19034111] (req 19030016->19034111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.778399:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:1.0:1789760717.779810:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971966 is committed 00000001:00080000:2.0:1789760717.815355:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971967 is committed 00002000:00010000:2.0:1789760717.815584:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19034112 -> 19038208 00010000:00010000:2.0:1789760717.820487:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54620 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.820508:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54620 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.820518:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54620 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.820525:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760717.820530:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54620 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.820540:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54620 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.820562:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54620 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.820570:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54620 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19034112->19038207] (req 19034112->19038207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.820853:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508272896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42071us (42457us total) trans 4294971967 rc 0/0 00000100:00100000:2.0:1789760717.820864:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4679 00000100:00100000:2.0:1789760717.820873:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696833796778 time 696883477332 state 0x2 ref 0 00000100:00100000:2.0:1789760717.826141:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760717.826229:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508273024 00000100:00100000:0.0:1789760717.826238:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508273024 00000100:00100000:0.0:1789760717.826388:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696833796778 time 696883477332 state 0x0 ref 1 00000100:00100000:0.0:1789760717.826403:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696883737748 time 696883477332 state 0x1 ref 1 00000100:00100000:0.0:1789760717.826422:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696883737748 time 696931133966 state 0x0 ref 1 00000100:00100000:0.0:1789760717.826433:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508273024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760717.826478:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19038208, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:0.0:1789760717.826487:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54627 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.826508:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=696s v=(0 0) 00010000:00010000:0.0:1789760717.826512:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760717.826518:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54627 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.826531:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54627 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760717.873731:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971968 is committed 00002000:00010000:2.0:1789760717.873920:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19038208 -> 19042304 00010000:00010000:2.0:1789760717.873937:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54627 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.873949:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54627 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.873957:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54627 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.873965:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760717.873970:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54627 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.873983:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54627 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.874000:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54627 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.874007:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54627 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19038208->19042303] (req 19038208->19042303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760717.874103:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508273024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47665us (47966us total) trans 4294971968 rc 0/0 00000100:00100000:2.0:1789760717.874112:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4680 00000100:00100000:2.0:1789760717.874122:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696883737748 time 696931133966 state 0x2 ref 0 00000100:00100000:2.0:1789760717.880224:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760717.912815:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971969 is committed 00000100:00100000:0.0:1789760717.916557:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760717.916805:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508273280 00000100:00100000:2.0:1789760717.916816:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508273280 00000100:00100000:2.0:1789760717.916870:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696931394382 time 696985419201 state 0x0 ref 1 00000100:00100000:2.0:1789760717.916886:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696985679617 time 696985419201 state 0x1 ref 1 00000100:00100000:2.0:1789760717.916919:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696985679617 time 697021629005 state 0x0 ref 1 00000100:00100000:2.0:1789760717.916941:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508273280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760717.916992:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19046400, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:2.0:1789760717.917002:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54635 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.917018:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760717.917022:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.917029:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54635 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.917042:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54635 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760717.970800:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971970 is committed 00002000:00010000:0.0:1789760717.970948:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19046400 -> 19050496 00010000:00010000:0.0:1789760717.970963:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54635 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.970976:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54635 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.970985:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54635 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.970992:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760717.970996:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54635 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.971005:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54635 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.971031:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54635 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760717.971040:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54635 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19046400->19050495] (req 19046400->19050495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760717.971133:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508273280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54198us (54581us total) trans 4294971970 rc 0/0 00000100:00100000:0.0:1789760717.971145:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4682 00000100:00100000:0.0:1789760717.971154:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696985679617 time 697021629005 state 0x2 ref 0 00000100:00100000:0.0:1789760717.975267:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760717.975411:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508273408 00000100:00100000:2.0:1789760717.975421:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508273408 00000100:00100000:2.0:1789760717.975487:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 696985679617 time 697021629005 state 0x0 ref 1 00000100:00100000:2.0:1789760717.975503:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697021889421 time 697021629005 state 0x1 ref 1 00000100:00100000:2.0:1789760717.975539:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697021889421 time 697080248848 state 0x0 ref 1 00000100:00100000:2.0:1789760717.975552:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508273408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760717.975615:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19050496, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:2.0:1789760717.975628:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5463c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.975643:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760717.975648:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760717.975654:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5463c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760717.975669:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5463c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760718.010327:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19050496 -> 19054592 00010000:00010000:2.0:1789760718.010347:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5463c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.010360:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5463c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.010368:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5463c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.010375:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760718.010381:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5463c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.010389:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5463c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.010409:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5463c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.010417:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5463c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19050496->19054591] (req 19050496->19054591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.010506:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508273408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34953us (35254us total) trans 4294971971 rc 0/0 00000100:00100000:2.0:1789760718.010514:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4683 00000100:00100000:2.0:1789760718.010522:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697021889421 time 697080248848 state 0x2 ref 0 00000100:00100000:2.0:1789760718.014428:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:2.0:1789760718.057990:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19054592 -> 19058688 00010000:00010000:2.0:1789760718.058004:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54643 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.058018:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54643 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.058027:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54643 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.058035:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760718.058040:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54643 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.058049:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54643 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.058068:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54643 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.058082:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54643 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19054592->19058687] (req 19054592->19058687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.058169:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508273536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43427us (43747us total) trans 4294971972 rc 0/0 00000100:00100000:2.0:1789760718.058178:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4684 00000100:00100000:2.0:1789760718.058188:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697080509264 time 697119439028 state 0x2 ref 0 00000100:00100000:0.0:1789760718.063823:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760718.063910:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508273664 00000100:00100000:2.0:1789760718.063920:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508273664 00000100:00100000:2.0:1789760718.063972:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697080509264 time 697119439028 state 0x0 ref 1 00000100:00100000:2.0:1789760718.063988:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697119699444 time 697119439028 state 0x1 ref 1 00000100:00100000:2.0:1789760718.064019:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697119699444 time 697168729165 state 0x0 ref 1 00000100:00100000:2.0:1789760718.064031:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508273664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760718.064078:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19058688, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:2.0:1789760718.064087:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5464a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.064103:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760718.064107:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.064113:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5464a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.064135:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5464a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760718.092329:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19058688 -> 19062784 00010000:00010000:0.0:1789760718.092344:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5464a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.092356:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5464a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.092365:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5464a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.092373:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760718.092378:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5464a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.092388:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5464a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.092406:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5464a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.092413:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5464a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19058688->19062783] (req 19058688->19062783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.092543:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508273664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28509us (28724us total) trans 4294971973 rc 0/0 00000100:00100000:0.0:1789760718.092553:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4685 00000100:00100000:0.0:1789760718.092562:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697119699444 time 697168729165 state 0x2 ref 0 00000100:00100000:0.0:1789760718.097110:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760718.097266:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508273792 00000100:00100000:2.0:1789760718.097276:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508273792 00000100:00100000:2.0:1789760718.097390:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697119699444 time 697168729165 state 0x0 ref 1 00000100:00100000:2.0:1789760718.097405:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697168989581 time 697168729165 state 0x1 ref 1 00000100:00100000:2.0:1789760718.097476:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697168989581 time 697202186776 state 0x0 ref 1 00000100:00100000:2.0:1789760718.097489:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508273792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760718.097544:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19062784, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:2.0:1789760718.097553:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54651 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.097569:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760718.097573:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.097579:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54651 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.097593:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54651 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760718.129658:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19062784 -> 19066880 00010000:00010000:2.0:1789760718.129681:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54651 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.129694:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54651 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.129702:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54651 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.129709:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760718.129713:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54651 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.129722:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54651 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.129742:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54651 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.129749:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54651 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19062784->19066879] (req 19062784->19066879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.129918:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508273792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32426us (32813us total) trans 4294971974 rc 0/0 00000100:00100000:2.0:1789760718.129928:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4686 00000100:00100000:2.0:1789760718.129938:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697168989581 time 697202186776 state 0x2 ref 0 00000100:00100000:2.0:1789760718.134759:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760718.134948:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971974 is committed 00002000:00010000:0.0:1789760718.176848:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19066880 -> 19070976 00010000:00010000:0.0:1789760718.176862:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54658 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.176874:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54658 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.176892:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54658 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.176903:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760718.176907:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54658 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.176916:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54658 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.176934:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54658 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.176941:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54658 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19066880->19070975] (req 19066880->19070975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.177036:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508273920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41599us (42282us total) trans 4294971975 rc 0/0 00000100:00100000:0.0:1789760718.177045:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4687 00000100:00100000:0.0:1789760718.177054:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697202447192 time 697240134026 state 0x2 ref 0 00000100:00100000:1.0:1789760718.184954:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760718.185211:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508274048 00000100:00100000:2.0:1789760718.185226:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508274048 00000100:00100000:2.0:1789760718.185417:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697202447192 time 697240134026 state 0x0 ref 1 00000100:00100000:2.0:1789760718.185439:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697240394442 time 697240134026 state 0x1 ref 1 00000100:00100000:2.0:1789760718.185470:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697240394442 time 697290179525 state 0x0 ref 1 00000100:00100000:2.0:1789760718.185483:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508274048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760718.185553:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19070976, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:2.0:1789760718.185562:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5465f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.185577:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760718.185581:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.185587:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5465f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.185600:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5465f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19070976->19075071] (req 19070976->19075071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.227591:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971976 is committed 00000100:00100000:2.0:1789760718.233012:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508274176 00000100:00100000:2.0:1789760718.233027:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508274176 00000100:00100000:2.0:1789760718.233082:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697240394442 time 697290179525 state 0x0 ref 1 00000100:00100000:2.0:1789760718.233098:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697290439941 time 697290179525 state 0x1 ref 1 00000100:00100000:2.0:1789760718.233132:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697290439941 time 697337842802 state 0x0 ref 1 00000100:00100000:2.0:1789760718.233143:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508274176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760718.233199:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19075072, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:2.0:1789760718.233208:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54666 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.233222:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760718.233226:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.233232:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54666 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.233256:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54666 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.267828:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971977 is committed 00002000:00010000:2.0:1789760718.268046:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19075072 -> 19079168 00010000:00010000:2.0:1789760718.268060:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54666 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.268073:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54666 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.268082:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54666 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.268089:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760718.268094:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54666 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.268104:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54666 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.268121:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54666 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.268128:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54666 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19075072->19079167] (req 19075072->19079167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.268419:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfed180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508274176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35273us (35619us total) trans 4294971977 rc 0/0 00000100:00100000:2.0:1789760718.268430:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4689 00000100:00100000:2.0:1789760718.268439:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697290439941 time 697337842802 state 0x2 ref 0 00000100:00100000:2.0:1789760718.273721:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760718.273830:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508274304 00000100:00100000:0.0:1789760718.273840:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508274304 00000100:00100000:0.0:1789760718.273893:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697290439941 time 697337842802 state 0x0 ref 1 00000100:00100000:0.0:1789760718.273907:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697338103218 time 697337842802 state 0x1 ref 1 00000100:00100000:0.0:1789760718.273983:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697338103218 time 697378692447 state 0x0 ref 1 00000100:00100000:0.0:1789760718.273996:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508274304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760718.274042:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19079168, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:0.0:1789760718.274051:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5466d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.274066:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760718.274071:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.274076:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5466d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.274089:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5466d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.342018:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971978 is committed 00002000:00010000:0.0:1789760718.342465:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19079168 -> 19083264 00010000:00010000:0.0:1789760718.342480:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5466d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.342491:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5466d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.342499:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5466d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.342505:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760718.342511:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5466d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.342521:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5466d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.342577:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5466d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.342704:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5466d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19079168->19083263] (req 19079168->19083263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.342802:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508274304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 68803us (69087us total) trans 4294971978 rc 0/0 00000100:00100000:0.0:1789760718.342813:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4690 00000100:00100000:0.0:1789760718.342822:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697338103218 time 697378692447 state 0x2 ref 0 00000100:00100000:0.0:1789760718.347283:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760718.347401:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508274432 00000100:00100000:2.0:1789760718.347410:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508274432 00000100:00100000:2.0:1789760718.347473:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697338103218 time 697378692447 state 0x0 ref 1 00000100:00100000:2.0:1789760718.347489:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697378952863 time 697378692447 state 0x1 ref 1 00000100:00100000:2.0:1789760718.347517:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697378952863 time 697452227454 state 0x0 ref 1 00000100:00100000:2.0:1789760718.347530:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508274432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760718.347577:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19083264, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:2.0:1789760718.347586:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54674 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.347611:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760718.347615:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.347621:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54674 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.347635:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54674 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760718.388490:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971979 is committed 00002000:00010000:0.0:1789760718.388717:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19083264 -> 19087360 00010000:00010000:0.0:1789760718.388732:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54674 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.388744:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54674 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.388754:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54674 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.388766:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760718.388786:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54674 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.388796:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54674 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.388815:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54674 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.388823:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54674 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19083264->19087359] (req 19083264->19087359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.389066:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508274432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41531us (41785us total) trans 4294971979 rc 0/0 00000100:00100000:0.0:1789760718.389077:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4691 00000100:00100000:0.0:1789760718.389086:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697378952863 time 697452227454 state 0x2 ref 0 00000100:00100000:1.0:1789760718.398018:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760718.398118:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508274560 00000100:00100000:2.0:1789760718.398129:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508274560 00000100:00100000:2.0:1789760718.398188:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697378952863 time 697452227454 state 0x0 ref 1 00000100:00100000:2.0:1789760718.398204:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697452487870 time 697452227454 state 0x1 ref 1 00000100:00100000:2.0:1789760718.398233:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697452487870 time 697502944348 state 0x0 ref 1 00000100:00100000:2.0:1789760718.398254:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508274560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760718.398348:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19087360, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:2.0:1789760718.398357:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5467b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.398373:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760718.398378:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.398384:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5467b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.398398:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5467b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760718.488706:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971980 is committed 00002000:00010000:2.0:1789760718.490816:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19087360 -> 19091456 00010000:00010000:2.0:1789760718.490835:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5467b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.490847:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5467b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.490856:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5467b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.490863:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760718.490868:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5467b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.490887:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5467b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.490907:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5467b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.490914:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5467b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19087360->19091455] (req 19087360->19091455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.492920:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508274560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 94665us (94900us total) trans 4294971980 rc 0/0 00000100:00100000:2.0:1789760718.492936:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4692 00000100:00100000:2.0:1789760718.492947:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697452487870 time 697502944348 state 0x2 ref 0 00000100:00100000:0.0:1789760718.504331:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508274688 00000100:00100000:0.0:1789760718.504343:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508274688 00000100:00100000:0.0:1789760718.504401:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697452487870 time 697502944348 state 0x0 ref 1 00000100:00100000:0.0:1789760718.504416:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697503204764 time 697502944348 state 0x1 ref 1 00000100:00100000:0.0:1789760718.504448:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697503204764 time 697609158735 state 0x0 ref 1 00000100:00100000:0.0:1789760718.504466:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508274688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760718.504514:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19091456, mtime: 0x6aad94ce, atime: 0x0, ctime: 0x6aad94ce, blocks: 0x9807 00010000:00010000:0.0:1789760718.504523:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54682 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.504538:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760718.504543:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.504549:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54682 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.504563:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54682 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760718.550148:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971981 is committed 00002000:00010000:0.0:1789760718.550387:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19091456 -> 19095552 00002000:00010000:0.0:1789760718.550392:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760718 -> 1789760719 00002000:00010000:0.0:1789760718.550394:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760718 -> 1789760719 00010000:00010000:0.0:1789760718.550408:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54682 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.550421:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54682 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.550429:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54682 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.550436:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760718.550440:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54682 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.550449:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54682 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.550469:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54682 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.550476:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54682 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19091456->19095551] (req 19091456->19095551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760718.550574:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508274688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46106us (47651us total) trans 4294971981 rc 0/0 00000100:00100000:0.0:1789760718.550585:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4693 00000100:00100000:0.0:1789760718.550658:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697503204764 time 697609158735 state 0x2 ref 0 00000100:00100000:1.0:1789760718.556127:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760718.556426:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508274816 00000100:00100000:0.0:1789760718.556436:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508274816 00000100:00100000:1.0:1789760718.556562:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697503204764 time 697609158735 state 0x0 ref 1 00000100:00100000:1.0:1789760718.556577:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697609419151 time 697609158735 state 0x1 ref 1 00000100:00100000:1.0:1789760718.556711:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697609419151 time 697661420444 state 0x0 ref 1 00000100:00100000:1.0:1789760718.556869:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508274816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760718.556989:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19095552, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:1.0:1789760718.556999:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54689 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.557015:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760718.557020:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.557026:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54689 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.557040:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54689 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.594445:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760718.595975:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971982 is committed 00002000:00010000:1.0:1789760718.596044:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19095552 -> 19099648 00010000:00010000:1.0:1789760718.596059:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54689 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.596071:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54689 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.596080:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54689 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.596091:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760718.596096:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54689 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.596106:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54689 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.596123:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54689 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.596130:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54689 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19095552->19099647] (req 19095552->19099647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.596387:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508274816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39658us (40264us total) trans 4294971982 rc 0/0 00000100:00100000:1.0:1789760718.596398:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4694 00000100:00100000:1.0:1789760718.596408:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697609419151 time 697661420444 state 0x2 ref 0 00000100:00100000:1.0:1789760718.600041:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760718.600124:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508274944 00000100:00100000:2.0:1789760718.600135:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508274944 00000100:00100000:2.0:1789760718.600197:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697609419151 time 697661420444 state 0x0 ref 1 00000100:00100000:2.0:1789760718.600213:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697661680860 time 697661420444 state 0x1 ref 1 00000100:00100000:2.0:1789760718.600252:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697661680860 time 697704954545 state 0x0 ref 1 00000100:00100000:2.0:1789760718.600264:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508274944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760718.600334:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19099648, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:2.0:1789760718.600344:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54690 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.600358:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:2.0:1789760718.600363:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.600369:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54690 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.600382:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54690 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.640464:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971983 is committed 00002000:00010000:1.0:1789760718.641176:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19099648 -> 19103744 00010000:00010000:1.0:1789760718.641191:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54690 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.641205:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54690 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.641218:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54690 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.641226:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760718.641230:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54690 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.641239:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54690 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.641271:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54690 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.641278:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54690 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19099648->19103743] (req 19099648->19103743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.641424:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508274944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41155us (41384us total) trans 4294971983 rc 0/0 00000100:00100000:1.0:1789760718.641435:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4695 00000100:00100000:1.0:1789760718.641446:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697661680860 time 697704954545 state 0x2 ref 0 00000100:00100000:1.0:1789760718.650219:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760718.650763:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508275072 00000100:00100000:0.0:1789760718.650773:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508275072 00000100:00100000:0.0:1789760718.650826:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697661680860 time 697704954545 state 0x0 ref 1 00000100:00100000:0.0:1789760718.650840:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697705214961 time 697704954545 state 0x1 ref 1 00000100:00100000:0.0:1789760718.650871:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697705214961 time 697755581752 state 0x0 ref 1 00000100:00100000:0.0:1789760718.650887:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508275072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760718.650944:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19103744, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:0.0:1789760718.650953:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54697 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.650968:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:0.0:1789760718.650972:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760718.650978:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54697 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760718.650991:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54697 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760718.685761:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19103744 -> 19107840 00010000:00010000:1.0:1789760718.685783:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54697 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.685801:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54697 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.685810:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54697 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.685817:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760718.685822:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54697 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.685832:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54697 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.685851:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54697 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.685860:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54697 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19103744->19107839] (req 19103744->19107839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.685970:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508275072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35079us (35754us total) trans 4294971984 rc 0/0 00000100:00100000:1.0:1789760718.685980:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4696 00000100:00100000:1.0:1789760718.685991:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697705214961 time 697755581752 state 0x2 ref 0 00000001:00080000:2.0:1789760718.686132:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971984 is committed 00000100:00100000:0.0:1789760718.690468:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760718.690648:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508275200 00000100:00100000:1.0:1789760718.690657:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508275200 00000100:00100000:1.0:1789760718.690705:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697705214961 time 697755581752 state 0x0 ref 1 00000100:00100000:1.0:1789760718.690719:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697755842168 time 697755581752 state 0x1 ref 1 00000100:00100000:1.0:1789760718.690787:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697755842168 time 697795497208 state 0x0 ref 1 00000100:00100000:1.0:1789760718.690799:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508275200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760718.690841:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19107840, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:1.0:1789760718.690849:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5469e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.690864:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760718.690868:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.690875:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5469e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.690888:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5469e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.718860:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04988d4ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534059648:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760718.719819:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760718.720508:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534059648 00000100:00100000:0.0:1789760718.720528:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534059648 00000100:00100000:0.0:1789760718.720598:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:0.0:1789760718.720608:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534059648:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760718.720650:0:7358:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789760718.720704:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534059648:12345-0@lo:41:osp-pre-1-0.0 Request processed in 97us (887us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760718.720710:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:2.0:1789760718.720846:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04988d4ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534059648:0@lo:41:osp-pre-1-0.0 00002000:00010000:1.0:1789760718.752090:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19107840 -> 19111936 00010000:00010000:1.0:1789760718.752107:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5469e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.752117:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5469e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.752127:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5469e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.752132:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760718.752135:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5469e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.752141:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5469e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.752155:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5469e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.752161:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5469e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19107840->19111935] (req 19107840->19111935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.752258:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508275200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61445us (61784us total) trans 4294971985 rc 0/0 00000100:00100000:1.0:1789760718.752269:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4697 00000100:00100000:1.0:1789760718.752278:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697755842168 time 697795497208 state 0x2 ref 0 00000001:00080000:0.0:1789760718.752376:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971985 is committed 00000100:00100000:0.0:1789760718.756354:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760718.756432:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508275328 00000100:00100000:1.0:1789760718.756441:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508275328 00000100:00100000:1.0:1789760718.756493:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697755842168 time 697795497208 state 0x0 ref 1 00000100:00100000:1.0:1789760718.756507:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697795757624 time 697795497208 state 0x1 ref 1 00000100:00100000:1.0:1789760718.756544:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697795757624 time 697861244087 state 0x0 ref 1 00000100:00100000:1.0:1789760718.756556:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508275328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760718.756613:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19111936, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:1.0:1789760718.756622:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.756636:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760718.756640:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.756646:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.756659:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760718.800656:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971986 is committed 00002000:00010000:1.0:1789760718.800821:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19111936 -> 19116032 00010000:00010000:1.0:1789760718.800837:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.800850:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.800859:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.800867:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760718.800873:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.800881:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.800900:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.800907:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19111936->19116031] (req 19111936->19116031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.801002:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508275328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44441us (44650us total) trans 4294971986 rc 0/0 00000100:00100000:1.0:1789760718.801012:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4698 00000100:00100000:1.0:1789760718.801022:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697795757624 time 697861244087 state 0x2 ref 0 00000100:00100000:1.0:1789760718.805221:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508275456 00000100:00100000:1.0:1789760718.805231:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508275456 00000100:00100000:1.0:1789760718.805344:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697795757624 time 697861244087 state 0x0 ref 1 00000100:00100000:1.0:1789760718.805359:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697861504503 time 697861244087 state 0x1 ref 1 00000100:00100000:1.0:1789760718.805393:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697861504503 time 697910102881 state 0x0 ref 1 00000100:00100000:1.0:1789760718.805406:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd9b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508275456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760718.805452:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19116032, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:1.0:1789760718.805462:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe546ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.805478:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760718.805483:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.805490:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe546ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.805504:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe546ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760718.839690:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19116032 -> 19120128 00010000:00010000:2.0:1789760718.839712:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe546ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.839726:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe546ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.839735:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe546ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.839742:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760718.839747:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe546ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.839756:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe546ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.839777:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe546ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.839790:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe546ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19116032->19120127] (req 19116032->19120127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.839885:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd9b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508275456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34476us (34769us total) trans 4294971987 rc 0/0 00000100:00100000:2.0:1789760718.839897:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4699 00000100:00100000:2.0:1789760718.839906:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697861504503 time 697910102881 state 0x2 ref 0 00000001:00080000:2.0:1789760718.839947:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971987 is committed 00000100:00100000:2.0:1789760718.844363:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760718.844463:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508275584 00000100:00100000:1.0:1789760718.844474:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508275584 00000100:00100000:1.0:1789760718.844529:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697861504503 time 697910102881 state 0x0 ref 1 00000100:00100000:1.0:1789760718.844546:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697910363297 time 697910102881 state 0x1 ref 1 00000100:00100000:1.0:1789760718.844614:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697910363297 time 697949324127 state 0x0 ref 1 00000100:00100000:1.0:1789760718.844628:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508275584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760718.844677:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19120128, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:1.0:1789760718.844686:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.844702:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=697s v=(0 0) 00010000:00010000:1.0:1789760718.844707:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.844713:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.844727:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760718.875079:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19120128 -> 19124224 00010000:00010000:2.0:1789760718.875100:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.875114:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.875123:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.875131:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760718.875136:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.875146:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.875178:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.875186:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe546b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19120128->19124223] (req 19120128->19124223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760718.875317:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508275584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30658us (30930us total) trans 4294971988 rc 0/0 00000100:00100000:2.0:1789760718.875327:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4700 00000100:00100000:2.0:1789760718.875336:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697910363297 time 697949324127 state 0x2 ref 0 00000100:00100000:0.0:1789760718.878259:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:1.0:1789760718.904958:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19124224 -> 19128320 00010000:00010000:1.0:1789760718.904977:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe546ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.904990:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe546ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.904999:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe546ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.905007:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760718.905012:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe546ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.905021:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe546ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.905042:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe546ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.905049:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe546ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19124224->19128319] (req 19124224->19128319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760718.905275:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508275712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26768us (27031us total) trans 4294971989 rc 0/0 00000100:00100000:1.0:1789760718.905326:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4701 00000100:00100000:1.0:1789760718.905335:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697949584543 time 697983201809 state 0x2 ref 0 00000100:00100000:2.0:1789760718.908778:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508275840 00000100:00100000:2.0:1789760718.908788:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508275840 00000100:00100000:2.0:1789760718.908840:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697949584543 time 697983201809 state 0x0 ref 1 00000100:00100000:2.0:1789760718.908854:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697983462225 time 697983201809 state 0x1 ref 1 00000100:00100000:2.0:1789760718.908883:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697983462225 time 698013593963 state 0x0 ref 1 00000100:00100000:2.0:1789760718.908895:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d44450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508275840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760718.908945:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19128320, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:2.0:1789760718.908954:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe546c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.908969:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760718.908975:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760718.908980:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe546c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760718.908994:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe546c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19128320->19132415] (req 19128320->19132415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760718.947977:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971990 is committed 00000100:00100000:1.0:1789760718.951973:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508275968 00000100:00100000:1.0:1789760718.951982:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508275968 00000100:00100000:1.0:1789760718.952034:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 697983462225 time 698013593963 state 0x0 ref 1 00000100:00100000:1.0:1789760718.952049:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698013854379 time 698013593963 state 0x1 ref 1 00000100:00100000:1.0:1789760718.952080:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698013854379 time 698056790139 state 0x0 ref 1 00000100:00100000:1.0:1789760718.952092:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508275968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760718.952138:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19132416, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:1.0:1789760718.952146:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe546c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.952173:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760718.952178:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760718.952184:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe546c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760718.952196:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe546c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760719.018075:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971991 is committed 00002000:00010000:1.0:1789760719.018337:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19132416 -> 19136512 00010000:00010000:1.0:1789760719.018352:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe546c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.018365:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe546c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.019278:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe546c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.019316:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760719.019321:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe546c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.019330:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe546c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.019351:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe546c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.019358:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe546c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19132416->19136511] (req 19132416->19136511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.019498:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508275968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 67401us (67624us total) trans 4294971991 rc 0/0 00000100:00100000:1.0:1789760719.019510:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4703 00000100:00100000:1.0:1789760719.019530:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698013854379 time 698056790139 state 0x2 ref 0 00000100:00100000:0.0:1789760719.024068:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760719.024139:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508276096 00000100:00100000:1.0:1789760719.024148:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508276096 00000100:00100000:1.0:1789760719.024224:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698013854379 time 698056790139 state 0x0 ref 1 00000100:00100000:1.0:1789760719.024239:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698057050555 time 698056790139 state 0x1 ref 1 00000100:00100000:1.0:1789760719.024283:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698057050555 time 698128992816 state 0x0 ref 1 00000100:00100000:1.0:1789760719.024325:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508276096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760719.024386:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19136512, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:1.0:1789760719.024395:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe546cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.024410:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760719.024414:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.024421:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe546cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.024434:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe546cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760719.024683:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760719.087281:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971992 is committed 00002000:00010000:0.0:1789760719.092153:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19136512 -> 19140608 00010000:00010000:0.0:1789760719.092176:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe546cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.092190:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe546cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.092198:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe546cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.092205:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760719.092210:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe546cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.092220:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe546cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.092249:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe546cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.092258:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe546cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19136512->19140607] (req 19136512->19140607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.092408:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508276096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 68080us (68344us total) trans 4294971992 rc 0/0 00000100:00100000:0.0:1789760719.092423:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4704 00000100:00100000:0.0:1789760719.092433:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698057050555 time 698128992816 state 0x2 ref 0 00000100:00100000:0.0:1789760719.100808:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760719.174194:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971993 is committed 00000100:00100000:1.0:1789760719.179439:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760719.179543:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508276352 00000100:00100000:2.0:1789760719.179553:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508276352 00000100:00100000:2.0:1789760719.179609:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698129253232 time 698205861324 state 0x0 ref 1 00000100:00100000:2.0:1789760719.179623:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698206121740 time 698205861324 state 0x1 ref 1 00000100:00100000:2.0:1789760719.179652:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698206121740 time 698284362507 state 0x0 ref 1 00000100:00100000:2.0:1789760719.179665:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508276352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760719.179713:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19144704, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:2.0:1789760719.179721:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe546dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.179737:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760719.179742:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.179748:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe546dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.179763:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe546dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760719.214902:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971994 is committed 00002000:00010000:2.0:1789760719.214966:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19144704 -> 19148800 00010000:00010000:2.0:1789760719.214982:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe546dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.214995:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe546dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.215004:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe546dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.215010:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760719.215014:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe546dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.215021:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe546dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.215038:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe546dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.215042:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe546dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19144704->19148799] (req 19144704->19148799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.215128:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508276352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35459us (35693us total) trans 4294971994 rc 0/0 00000100:00100000:2.0:1789760719.215142:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4706 00000100:00100000:2.0:1789760719.215151:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698206121740 time 698284362507 state 0x2 ref 0 00000100:00100000:0.0:1789760719.231718:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760719.264141:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971995 is committed 00000100:00100000:0.0:1789760719.267611:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760719.300892:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971996 is committed 00002000:00010000:1.0:1789760719.301267:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19152896 -> 19156992 00010000:00010000:1.0:1789760719.301326:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe546eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.301342:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe546eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.301352:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe546eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.301360:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760719.301366:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe546eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.301376:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe546eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.301401:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe546eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.301410:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe546eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19152896->19156991] (req 19152896->19156991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.301534:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508276608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33246us (33933us total) trans 4294971996 rc 0/0 00000100:00100000:1.0:1789760719.301546:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4708 00000100:00100000:1.0:1789760719.301558:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698336918362 time 698372978421 state 0x2 ref 0 00000100:00100000:2.0:1789760719.307172:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760719.307230:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508276736 00000100:00100000:1.0:1789760719.307255:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508276736 00000100:00100000:1.0:1789760719.307383:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698336918362 time 698372978421 state 0x0 ref 1 00000100:00100000:1.0:1789760719.307402:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698373238837 time 698372978421 state 0x1 ref 1 00000100:00100000:1.0:1789760719.307427:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698373238837 time 698412137200 state 0x0 ref 1 00000100:00100000:1.0:1789760719.307440:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508276736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760719.307503:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19156992, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:1.0:1789760719.307515:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe546f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.307537:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760719.307543:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.307550:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe546f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.307569:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe546f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.335620:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971997 is committed 00002000:00010000:1.0:1789760719.335803:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19156992 -> 19161088 00010000:00010000:1.0:1789760719.335824:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe546f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.335841:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe546f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.335851:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe546f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.335859:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760719.335865:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe546f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.335875:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe546f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.335900:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe546f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.335908:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe546f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19156992->19161087] (req 19156992->19161087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.336035:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508276736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28590us (28870us total) trans 4294971997 rc 0/0 00000100:00100000:1.0:1789760719.336047:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4709 00000100:00100000:1.0:1789760719.336059:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698373238837 time 698412137200 state 0x2 ref 0 00000100:00100000:0.0:1789760719.338768:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760719.371896:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19161088 -> 19165184 00010000:00010000:0.0:1789760719.371926:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe546f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.371943:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe546f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.371953:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe546f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.371960:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760719.371966:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe546f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.371977:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe546f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.372012:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe546f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.372012:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971998 is committed 00010000:00010000:0.0:1789760719.372020:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe546f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19161088->19165183] (req 19161088->19165183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.372136:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508276864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33093us (33376us total) trans 4294971998 rc 0/0 00000100:00100000:0.0:1789760719.372148:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4710 00000100:00100000:0.0:1789760719.372162:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698412397616 time 698443734101 state 0x2 ref 0 00000100:00100000:0.0:1789760719.375761:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760719.375930:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508276992 00000100:00100000:1.0:1789760719.375944:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508276992 00000100:00100000:1.0:1789760719.376016:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698412397616 time 698443734101 state 0x0 ref 1 00000100:00100000:1.0:1789760719.376036:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698443994517 time 698443734101 state 0x1 ref 1 00000100:00100000:1.0:1789760719.376087:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698443994517 time 698480797935 state 0x0 ref 1 00000100:00100000:1.0:1789760719.376097:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508276992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760719.376172:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19165184, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:1.0:1789760719.376182:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54700 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.376203:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760719.376207:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.376215:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54700 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19165184->19169279] (req 19165184->19169279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.376232:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54700 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19165184->19169279] (req 19165184->19169279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760719.419634:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294971999 is committed 00000100:00100000:2.0:1789760719.423473:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760719.423854:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508277120 00000100:00100000:1.0:1789760719.423863:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508277120 00000100:00100000:1.0:1789760719.423913:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698443994517 time 698480797935 state 0x0 ref 1 00000100:00100000:1.0:1789760719.423928:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698481058351 time 698480797935 state 0x1 ref 1 00000100:00100000:1.0:1789760719.423959:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698481058351 time 698528668941 state 0x0 ref 1 00000100:00100000:1.0:1789760719.423969:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508277120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760719.424009:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19169280, mtime: 0x6aad94cf, atime: 0x0, ctime: 0x6aad94cf, blocks: 0x9807 00010000:00010000:1.0:1789760719.424017:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54707 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.424029:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760719.424032:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.424036:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54707 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.424046:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54707 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760719.467231:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19169280 -> 19173376 00010000:00010000:2.0:1789760719.467260:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54707 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.467272:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54707 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.467281:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54707 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.467313:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760719.467318:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54707 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.467326:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54707 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.467346:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54707 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.467360:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54707 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19169280->19173375] (req 19169280->19173375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.467454:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508277120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43480us (43983us total) trans 4294972000 rc 0/0 00000100:00100000:2.0:1789760719.467464:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4712 00000100:00100000:2.0:1789760719.467473:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698481058351 time 698528668941 state 0x2 ref 0 00000001:00080000:0.0:1789760719.468509:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972000 is committed 00000100:00100000:0.0:1789760719.471216:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:1.0:1789760719.517375:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972001 is committed 00002000:00010000:0.0:1789760719.517598:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19173376 -> 19177472 00002000:00010000:0.0:1789760719.517603:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760719 -> 1789760720 00002000:00010000:0.0:1789760719.517606:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760719 -> 1789760720 00010000:00010000:0.0:1789760719.517618:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5470e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.517631:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5470e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.517640:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5470e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.517647:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760719.517652:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5470e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.517660:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5470e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.517679:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5470e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.517686:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5470e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19173376->19177471] (req 19173376->19177471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.517786:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508277248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46120us (46573us total) trans 4294972001 rc 0/0 00000100:00100000:0.0:1789760719.517795:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4713 00000100:00100000:0.0:1789760719.517805:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698528929357 time 698576359093 state 0x2 ref 0 00000100:00100000:1.0:1789760719.524968:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:1.0:1789760719.600054:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19177472 -> 19181568 00010000:00010000:1.0:1789760719.600074:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54715 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.600088:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54715 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.600097:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54715 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.600105:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760719.600109:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54715 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.600123:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54715 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.600142:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54715 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.600150:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54715 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19177472->19181567] (req 19177472->19181567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.600260:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508277376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 75083us (75295us total) trans 4294972002 rc 0/0 00000100:00100000:1.0:1789760719.600271:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4714 00000100:00100000:1.0:1789760719.600281:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698576619509 time 698629869255 state 0x2 ref 0 00000001:00080000:1.0:1789760719.600382:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972002 is committed 00000100:00100000:2.0:1789760719.605878:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760719.607750:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508277504 00000100:00100000:1.0:1789760719.607764:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508277504 00000100:00100000:1.0:1789760719.607823:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698576619509 time 698629869255 state 0x0 ref 1 00000100:00100000:1.0:1789760719.607839:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698630129671 time 698629869255 state 0x1 ref 1 00000100:00100000:1.0:1789760719.607856:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698630129671 time 698712567179 state 0x0 ref 1 00000100:00100000:1.0:1789760719.607870:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508277504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760719.607918:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19181568, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:1.0:1789760719.607929:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5471c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.607946:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760719.607951:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.607957:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5471c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.607971:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5471c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.614762:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760719.661507:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19181568 -> 19185664 00010000:00010000:1.0:1789760719.661546:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5471c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.661559:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5471c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.661569:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5471c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.661576:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760719.661580:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5471c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.661596:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5471c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.661622:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5471c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.661630:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5471c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19181568->19185663] (req 19181568->19185663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.661937:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508277504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54063us (56061us total) trans 4294972003 rc 0/0 00000100:00100000:1.0:1789760719.661949:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4715 00000100:00100000:1.0:1789760719.661960:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698630129671 time 698712567179 state 0x2 ref 0 00000001:00080000:1.0:1789760719.662004:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972003 is committed 00000100:00100000:0.0:1789760719.665208:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760719.697024:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972004 is committed 00000100:00100000:2.0:1789760719.701841:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760719.755333:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19189760 -> 19193856 00010000:00010000:0.0:1789760719.755349:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5472a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.755362:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5472a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.755377:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5472a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.755385:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760719.755390:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5472a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.755399:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5472a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.755418:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5472a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.755426:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5472a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19189760->19193855] (req 19189760->19193855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.755534:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508277760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52829us (53697us total) trans 4294972005 rc 0/0 00000100:00100000:0.0:1789760719.755545:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4717 00000100:00100000:0.0:1789760719.755556:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698770456202 time 698807399194 state 0x2 ref 0 00000100:00100000:2.0:1789760719.758696:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760719.759202:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508277888 00000100:00100000:1.0:1789760719.759213:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508277888 00000100:00100000:1.0:1789760719.759277:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698770456202 time 698807399194 state 0x0 ref 1 00000100:00100000:1.0:1789760719.759327:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698807659610 time 698807399194 state 0x1 ref 1 00000100:00100000:1.0:1789760719.759358:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698807659610 time 698864068581 state 0x0 ref 1 00000100:00100000:1.0:1789760719.759371:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508277888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760719.759420:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19193856, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:1.0:1789760719.759430:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54731 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.759451:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:1.0:1789760719.759456:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.759464:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54731 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.759478:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54731 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760719.797811:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19193856 -> 19197952 00010000:00010000:1.0:1789760719.797832:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54731 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.797845:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54731 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.797854:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54731 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.797861:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760719.797865:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54731 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.797875:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54731 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.797894:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54731 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.797899:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54731 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19193856->19197951] (req 19193856->19197951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.797976:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508277888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38603us (39287us total) trans 4294972006 rc 0/0 00000100:00100000:1.0:1789760719.797986:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4718 00000100:00100000:1.0:1789760719.797995:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698807659610 time 698864068581 state 0x2 ref 0 00000100:00100000:2.0:1789760719.802104:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760719.836627:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972007 is committed 00000100:00100000:2.0:1789760719.841833:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760719.841937:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508278144 00000100:00100000:0.0:1789760719.841947:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508278144 00000100:00100000:0.0:1789760719.842013:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698864328997 time 698907361361 state 0x0 ref 1 00000100:00100000:0.0:1789760719.842030:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698907621777 time 698907361361 state 0x1 ref 1 00000100:00100000:0.0:1789760719.842063:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698907621777 time 698946772610 state 0x0 ref 1 00000100:00100000:0.0:1789760719.842075:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508278144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760719.842124:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19202048, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:0.0:1789760719.842133:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5473f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.842149:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:0.0:1789760719.842154:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.842160:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5473f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.842175:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5473f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.866952:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972008 is committed 00002000:00010000:1.0:1789760719.866971:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19202048 -> 19206144 00010000:00010000:1.0:1789760719.866993:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5473f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.867007:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5473f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.867016:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5473f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.867024:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760719.867029:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5473f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.867038:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5473f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.867065:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5473f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.867074:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5473f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19202048->19206143] (req 19202048->19206143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.867161:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508278144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25084us (25335us total) trans 4294972008 rc 0/0 00000100:00100000:1.0:1789760719.867172:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4720 00000100:00100000:1.0:1789760719.867183:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698907621777 time 698946772610 state 0x2 ref 0 00000100:00100000:1.0:1789760719.870256:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760719.870429:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508278272 00000100:00100000:2.0:1789760719.870439:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508278272 00000100:00100000:2.0:1789760719.870490:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698907621777 time 698946772610 state 0x0 ref 1 00000100:00100000:2.0:1789760719.870504:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698947033026 time 698946772610 state 0x1 ref 1 00000100:00100000:2.0:1789760719.870533:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698947033026 time 698975243912 state 0x0 ref 1 00000100:00100000:2.0:1789760719.870546:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508278272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760719.870592:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19206144, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:2.0:1789760719.870601:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54746 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.870615:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=698s v=(0 0) 00010000:00010000:2.0:1789760719.870619:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.870638:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54746 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.870653:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54746 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19206144->19210239] (req 19206144->19210239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760719.895044:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972009 is committed 00000100:00100000:2.0:1789760719.899354:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760719.918037:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19210240 -> 19214336 00010000:00010000:0.0:1789760719.918052:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5474d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.918063:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5474d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.918071:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5474d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.918077:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760719.918082:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5474d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.918130:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5474d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.918150:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5474d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.918158:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5474d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19210240->19214335] (req 19210240->19214335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760719.918374:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508278400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 18800us (19023us total) trans 4294972010 rc 0/0 00000100:00100000:0.0:1789760719.918384:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4722 00000100:00100000:0.0:1789760719.918541:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698975504328 time 699004270504 state 0x2 ref 0 00000100:00100000:2.0:1789760719.920915:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760719.921009:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508278528 00000100:00100000:1.0:1789760719.921019:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508278528 00000100:00100000:1.0:1789760719.921069:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 698975504328 time 699004270504 state 0x0 ref 1 00000100:00100000:1.0:1789760719.921084:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699004530920 time 699004270504 state 0x1 ref 1 00000100:00100000:1.0:1789760719.921110:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699004530920 time 699025820403 state 0x0 ref 1 00000100:00100000:1.0:1789760719.921123:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508278528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760719.921464:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19214336, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:1.0:1789760719.921475:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54754 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.921499:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760719.921504:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760719.921510:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54754 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.921526:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54754 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760719.936741:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760719.936781:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760719.937124:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508278784 00000100:00100000:2.0:1789760719.937136:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760719.937138:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508278784 00000100:00100000:2.0:1789760719.937165:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760719.937198:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 849 00000100:00100000:2.0:1789760719.937198:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760719.937212:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2407:x1876699508278784:12345-192.168.203.23@tcp:400:kworker.0 00000100:00080000:2.0:1789760719.937220:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760719.937232:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508279040 00000100:00100000:2.0:1789760719.937249:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508279040 00000100:00100000:2.0:1789760719.937281:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508278912 00000100:00100000:2.0:1789760719.937305:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508278912 00000100:00100000:2.0:1789760719.937322:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508279168 00000100:00100000:1.0:1789760719.937325:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2407:x1876699508278784:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 114us (591us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760719.937325:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508279168 00000100:00100000:1.0:1789760719.937336:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 849 00000100:00100000:0.0:1789760719.937388:0:6459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508278656 00000100:00100000:0.0:1789760719.937393:0:6459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508278656 00000100:00100000:0.0:1789760719.937426:0:6459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 463 00000100:00100000:0.0:1789760719.937434:0:6459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04989030c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2406:x1876699508278656:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760719.937465:0:6459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04989030c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2406:x1876699508278656:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 32us (330us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760719.937471:0:6459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 463 00000100:00100000:2.0:1789760719.938274:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 850 00000100:00100000:0.0:1789760719.938350:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 851 00000100:00100000:0.0:1789760719.938360:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508278912:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760719.938385:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 852 00000100:00100000:0.0:1789760719.938405:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508278912:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 46us (1239us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760719.938414:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 851 00000100:00100000:2.0:1789760719.938468:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508279040:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760719.938504:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508279040:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 221us (1722us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760719.938512:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 850 00000100:00100000:1.0:1789760719.938693:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2406:x1876699508279168:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760719.938736:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2406:x1876699508279168:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 48us (1539us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760719.938743:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 852 00000100:00080000:2.0:1789760719.939141:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760719.939152:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789760719.947578:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972011 is committed 00002000:00010000:1.0:1789760719.947661:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19214336 -> 19218432 00010000:00010000:1.0:1789760719.947677:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54754 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.947691:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54754 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.947697:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54754 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.947704:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760719.947709:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54754 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.947718:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54754 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.947737:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54754 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760719.947745:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54754 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19214336->19218431] (req 19214336->19218431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760719.947919:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508278528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26755us (26970us total) trans 4294972011 rc 0/0 00000100:00100000:1.0:1789760719.947929:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4723 00000100:00100000:1.0:1789760719.947938:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699004530920 time 699025820403 state 0x2 ref 0 00000100:00100000:2.0:1789760719.956630:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508279296 00000100:00100000:2.0:1789760719.956640:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508279296 00000100:00100000:2.0:1789760719.956693:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699004530920 time 699025820403 state 0x0 ref 1 00000100:00100000:2.0:1789760719.956708:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699026080819 time 699025820403 state 0x1 ref 1 00000100:00100000:2.0:1789760719.956741:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699026080819 time 699061452223 state 0x0 ref 1 00000100:00100000:2.0:1789760719.956754:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd9b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508279296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760719.956797:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19218432, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:2.0:1789760719.956805:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5475b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.956820:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760719.956825:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760719.956831:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5475b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760719.956845:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5475b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19218432->19222527] (req 19218432->19222527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760719.993054:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972012 is committed 00000100:00100000:2.0:1789760719.994819:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760719.995011:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508279424 00000100:00100000:0.0:1789760719.995022:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508279424 00000100:00100000:0.0:1789760719.995077:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699026080819 time 699061452223 state 0x0 ref 1 00000100:00100000:0.0:1789760719.995092:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699061712639 time 699061452223 state 0x1 ref 1 00000100:00100000:0.0:1789760719.995124:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699061712639 time 699099834899 state 0x0 ref 1 00000100:00100000:0.0:1789760719.995135:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508279424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760719.995190:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19222528, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:0.0:1789760719.995199:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54762 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.995216:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760719.995220:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760719.995227:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54762 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19222528->19226623] (req 19222528->19226623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760719.995252:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54762 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19222528->19226623] (req 19222528->19226623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.017180:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972013 is committed 00000100:00100000:2.0:1789760720.021679:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760720.021754:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508279552 00000100:00100000:1.0:1789760720.021763:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508279552 00000100:00100000:1.0:1789760720.021810:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699061712639 time 699099834899 state 0x0 ref 1 00000100:00100000:1.0:1789760720.021822:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699100095315 time 699099834899 state 0x1 ref 1 00000100:00100000:1.0:1789760720.021845:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699100095315 time 699126555942 state 0x0 ref 1 00000100:00100000:1.0:1789760720.021858:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508279552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760720.021907:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19226624, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:1.0:1789760720.021914:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54769 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.021928:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760720.021932:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.021936:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54769 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.021947:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54769 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19226624->19230719] (req 19226624->19230719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.041080:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972014 is committed 00000100:00100000:0.0:1789760720.044217:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760720.065355:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972015 is committed 00000100:00100000:2.0:1789760720.070611:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760720.089865:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972016 is committed 00002000:00010000:0.0:1789760720.089972:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19234816 -> 19238912 00010000:00010000:0.0:1789760720.089988:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54777 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.090001:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54777 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.090011:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54777 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.090019:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760720.090023:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54777 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.090033:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54777 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.090053:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54777 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.090060:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54777 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19234816->19238911] (req 19234816->19238911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.090192:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508279808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19380us (19586us total) trans 4294972016 rc 0/0 00000100:00100000:0.0:1789760720.090204:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4728 00000100:00100000:0.0:1789760720.090213:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699149911498 time 699175451627 state 0x2 ref 0 00000100:00100000:2.0:1789760720.099141:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760720.118761:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972017 is committed 00000100:00100000:0.0:1789760720.127230:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508280064 00000100:00100000:0.0:1789760720.127253:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508280064 00000100:00100000:0.0:1789760720.127355:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699175712043 time 699204099426 state 0x0 ref 1 00000100:00100000:0.0:1789760720.127370:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699204359842 time 699204099426 state 0x1 ref 1 00000100:00100000:0.0:1789760720.127391:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699204359842 time 699232102281 state 0x0 ref 1 00000100:00100000:0.0:1789760720.127405:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508280064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760720.127453:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19243008, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:0.0:1789760720.127462:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54785 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.127478:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760720.127483:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.127489:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54785 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.127502:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54785 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19243008->19247103] (req 19243008->19247103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.194809:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972018 is committed 00000100:00100000:2.0:1789760720.199082:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760720.199371:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508280192 00000100:00100000:0.0:1789760720.199382:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508280192 00000100:00100000:0.0:1789760720.199440:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699204359842 time 699232102281 state 0x0 ref 1 00000100:00100000:0.0:1789760720.199456:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699232362697 time 699232102281 state 0x1 ref 1 00000100:00100000:0.0:1789760720.199527:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699232362697 time 699304237432 state 0x0 ref 1 00000100:00100000:0.0:1789760720.199541:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508280192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760720.199598:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19247104, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:0.0:1789760720.199607:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5478c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.199623:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760720.199628:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.199634:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5478c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.199648:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5478c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760720.236351:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972019 is committed 00002000:00010000:0.0:1789760720.236473:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19247104 -> 19251200 00010000:00010000:0.0:1789760720.236489:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5478c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.236501:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5478c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.236509:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5478c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.236516:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760720.236521:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5478c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.236530:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5478c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.236797:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5478c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.236807:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5478c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19247104->19251199] (req 19247104->19251199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.236938:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508280192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37394us (37860us total) trans 4294972019 rc 0/0 00000100:00100000:0.0:1789760720.236949:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4731 00000100:00100000:0.0:1789760720.236959:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699232362697 time 699304237432 state 0x2 ref 0 00000100:00100000:2.0:1789760720.247851:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760720.280994:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972020 is committed 00000100:00100000:2.0:1789760720.292182:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760720.319614:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19255296 -> 19259392 00010000:00010000:0.0:1789760720.319629:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5479a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.319642:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5479a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.319651:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5479a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.319665:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760720.319669:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5479a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.319678:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5479a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.319697:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5479a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.319704:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5479a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19255296->19259391] (req 19255296->19259391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760720.319826:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508280448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27267us (27649us total) trans 4294972021 rc 0/0 00000100:00100000:0.0:1789760720.319837:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4733 00000100:00100000:0.0:1789760720.319847:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699353019982 time 699397251929 state 0x2 ref 0 00000100:00100000:0.0:1789760720.329033:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:1.0:1789760720.349887:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19259392 -> 19263488 00010000:00010000:1.0:1789760720.349910:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe547a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.349924:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe547a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.349931:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972022 is committed 00010000:00010000:1.0:1789760720.349937:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe547a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.349943:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760720.349948:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe547a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.349958:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe547a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.349978:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe547a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.349985:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe547a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19259392->19263487] (req 19259392->19263487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.350281:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508280576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20666us (21253us total) trans 4294972022 rc 0/0 00000100:00100000:1.0:1789760720.350327:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4734 00000100:00100000:1.0:1789760720.350336:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699397512345 time 699434305993 state 0x2 ref 0 00000100:00100000:2.0:1789760720.358743:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760720.387731:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972023 is committed 00000100:00100000:2.0:1789760720.400486:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760720.400777:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508280832 00000100:00100000:0.0:1789760720.400788:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508280832 00000100:00100000:0.0:1789760720.400849:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699434566409 time 699463648417 state 0x0 ref 1 00000100:00100000:0.0:1789760720.400865:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699463908833 time 699463648417 state 0x1 ref 1 00000100:00100000:0.0:1789760720.400943:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699463908833 time 699505652802 state 0x0 ref 1 00000100:00100000:0.0:1789760720.400957:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508280832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760720.401010:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19267584, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:0.0:1789760720.401019:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe547af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.401036:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:0.0:1789760720.401040:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760720.401047:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe547af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760720.401061:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe547af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760720.454340:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19267584 -> 19271680 00010000:00010000:1.0:1789760720.454356:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe547af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.454375:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe547af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.454384:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe547af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.454392:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760720.454396:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe547af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.454404:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe547af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.454427:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe547af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.454447:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe547af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19267584->19271679] (req 19267584->19271679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.454549:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508280832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53589us (54068us total) trans 4294972024 rc 0/0 00000100:00100000:1.0:1789760720.454559:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4736 00000100:00100000:1.0:1789760720.454569:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699463908833 time 699505652802 state 0x2 ref 0 00000100:00100000:0.0:1789760720.458626:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760720.458741:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508280960 00000100:00100000:1.0:1789760720.458752:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508280960 00000100:00100000:1.0:1789760720.458805:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699463908833 time 699505652802 state 0x0 ref 1 00000100:00100000:1.0:1789760720.458819:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699505913218 time 699505652802 state 0x1 ref 1 00000100:00100000:1.0:1789760720.458848:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699505913218 time 699563558429 state 0x0 ref 1 00000100:00100000:1.0:1789760720.458860:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508280960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760720.458909:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19271680, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:1.0:1789760720.458918:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.458935:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760720.458939:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.458946:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.458961:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760720.532188:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19271680 -> 19275776 00010000:00010000:1.0:1789760720.532204:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.532216:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.532239:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.532247:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760720.532251:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.532261:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.532281:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.532317:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19271680->19275775] (req 19271680->19275775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.532404:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508280960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 73541us (73784us total) trans 4294972025 rc 0/0 00000100:00100000:1.0:1789760720.532414:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4737 00000100:00100000:1.0:1789760720.532425:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699505913218 time 699563558429 state 0x2 ref 0 00000100:00100000:1.0:1789760720.539180:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508281088 00000100:00100000:1.0:1789760720.539190:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508281088 00000100:00100000:1.0:1789760720.539255:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699505913218 time 699563558429 state 0x0 ref 1 00000100:00100000:1.0:1789760720.539270:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699563818845 time 699563558429 state 0x1 ref 1 00000100:00100000:1.0:1789760720.539318:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699563818845 time 699644027993 state 0x0 ref 1 00000100:00100000:1.0:1789760720.539330:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508281088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760720.539376:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19275776, mtime: 0x6aad94d0, atime: 0x0, ctime: 0x6aad94d0, blocks: 0x9807 00010000:00010000:1.0:1789760720.539385:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe547bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.539400:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760720.539404:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.539410:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe547bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.539425:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe547bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.608539:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972026 is committed 00002000:00010000:1.0:1789760720.608649:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19275776 -> 19279872 00002000:00010000:1.0:1789760720.608653:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760720 -> 1789760721 00002000:00010000:1.0:1789760720.608656:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760720 -> 1789760721 00010000:00010000:1.0:1789760720.608668:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe547bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.608682:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe547bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.608691:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe547bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.608699:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760720.608704:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe547bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.608713:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe547bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.608732:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe547bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.608740:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe547bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19275776->19279871] (req 19275776->19279871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.608831:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508281088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 69499us (69797us total) trans 4294972026 rc 0/0 00000100:00100000:1.0:1789760720.608841:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4738 00000100:00100000:1.0:1789760720.608851:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699563818845 time 699644027993 state 0x2 ref 0 00000100:00100000:1.0:1789760720.612259:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508281216 00000100:00100000:1.0:1789760720.612269:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508281216 00000100:00100000:1.0:1789760720.612349:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699563818845 time 699644027993 state 0x0 ref 1 00000100:00100000:1.0:1789760720.612364:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699644288409 time 699644027993 state 0x1 ref 1 00000100:00100000:1.0:1789760720.612383:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699644288409 time 699717094476 state 0x0 ref 1 00000100:00100000:1.0:1789760720.612398:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508281216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760720.612445:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19279872, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9807 00010000:00010000:1.0:1789760720.612453:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.612467:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760720.612472:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.612477:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.612490:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760720.655339:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19279872 -> 19283968 00010000:00010000:1.0:1789760720.655358:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.655371:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.655379:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.655386:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760720.655390:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.655400:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.655418:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.655425:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19279872->19283967] (req 19279872->19283967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.655507:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508281216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43106us (43431us total) trans 4294972027 rc 0/0 00000100:00100000:1.0:1789760720.655517:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4739 00000100:00100000:1.0:1789760720.655526:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699644288409 time 699717094476 state 0x2 ref 0 00000001:00080000:1.0:1789760720.655567:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972027 is committed 00000001:00080000:0.0:1789760720.708332:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972028 is committed 00002000:00010000:1.0:1789760720.708476:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19283968 -> 19288064 00010000:00010000:1.0:1789760720.708491:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe547cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.708503:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe547cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.708512:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe547cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.708523:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760720.708528:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe547cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.708537:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe547cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.708556:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe547cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.708564:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe547cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19283968->19288063] (req 19283968->19288063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.708660:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508281344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48521us (48692us total) trans 4294972028 rc 0/0 00000100:00100000:1.0:1789760720.708670:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4740 00000100:00100000:1.0:1789760720.708680:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699717354892 time 699764833660 state 0x2 ref 0 00000100:00100000:1.0:1789760720.712134:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508281472 00000100:00100000:1.0:1789760720.712153:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508281472 00000100:00100000:1.0:1789760720.712207:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699717354892 time 699764833660 state 0x0 ref 1 00000100:00100000:1.0:1789760720.712221:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699765094076 time 699764833660 state 0x1 ref 1 00000100:00100000:1.0:1789760720.712255:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699765094076 time 699816966275 state 0x0 ref 1 00000100:00100000:1.0:1789760720.712270:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508281472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760720.712342:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19288064, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9807 00010000:00010000:1.0:1789760720.712350:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.712365:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760720.712369:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.712374:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.712388:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.736847:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972029 is committed 00002000:00010000:2.0:1789760720.736964:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19288064 -> 19292160 00010000:00010000:2.0:1789760720.736981:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.736993:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.737002:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.737015:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760720.737020:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.737029:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.737048:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.737055:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe547d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19288064->19292159] (req 19288064->19292159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.737214:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508281472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24869us (25091us total) trans 4294972029 rc 0/0 00000100:00100000:2.0:1789760720.737225:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4741 00000100:00100000:2.0:1789760720.737244:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699765094076 time 699816966275 state 0x2 ref 0 00000100:00100000:1.0:1789760720.744752:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508281600 00000100:00100000:1.0:1789760720.744763:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508281600 00000100:00100000:1.0:1789760720.744821:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699765094076 time 699816966275 state 0x0 ref 1 00000100:00100000:1.0:1789760720.744836:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699817226691 time 699816966275 state 0x1 ref 1 00000100:00100000:1.0:1789760720.744859:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699817226691 time 699849570084 state 0x0 ref 1 00000100:00100000:1.0:1789760720.744871:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508281600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760720.744918:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19292160, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9807 00010000:00010000:1.0:1789760720.744926:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe547d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.744946:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:1.0:1789760720.744950:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.744955:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe547d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.744968:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe547d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.766594:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760720.766615:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b6c2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534059776:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760720.766682:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760720.768003:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760720.768014:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b6c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534059904:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760720.768054:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760720.768089:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b6c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534060160:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760720.768110:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760720.768126:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760720.768130:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b6c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534060032:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760720.768162:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760720.768185:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534059904 00000100:00100000:1.0:1789760720.768191:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534059904 00000100:00100000:1.0:1789760720.768218:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534060160 00000100:00100000:1.0:1789760720.768220:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534060160 00000100:00100000:1.0:1789760720.768242:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534060032 00000100:00100000:1.0:1789760720.768244:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534060032 00000100:00100000:1.0:1789760720.768257:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 353 00000100:00100000:1.0:1789760720.768263:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534059904:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760720.768278:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760720.768331:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534059904:12345-0@lo:13:osp-pre-1-1.0 Request processed in 67us (276us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760720.768338:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 353 00000100:00100000:1.0:1789760720.768354:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:1.0:1789760720.768360:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534060160:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760720.768369:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3838836736 avail 3794796544 00000100:00100000:1.0:1789760720.768394:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534060160:12345-0@lo:13:osp-pre-0-0.0 Request processed in 34us (283us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760720.768400:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00100000:1.0:1789760720.768410:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:1.0:1789760720.768415:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534060032:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760720.768423:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3838836736 avail 3794796544 00000100:00100000:1.0:1789760720.768441:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534060032:12345-0@lo:13:osp-pre-0-1.0 Request processed in 27us (279us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760720.768446:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00080000:1.0:1789760720.768459:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789760720.768462:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760720.768484:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b6c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534059904:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760720.768512:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b6c0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534060032:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760720.768523:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b6c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534060160:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760720.769095:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b645c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534060544:0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.769133:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760720.769472:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3472:1876699534061056:0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.769501:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760720.769524:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1789760720.769529:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b645500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534060672:0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.769552:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760720.769567:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b644e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3472:1876699534061184:0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.769602:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760720.769640:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534061056 00000100:00100000:0.0:1789760720.769646:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534061056 00000100:00100000:0.0:1789760720.769682:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534060672 00000100:00100000:0.0:1789760720.769685:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534060672 00000100:00100000:0.0:1789760720.769702:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534061184 00000100:00100000:0.0:1789760720.769705:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534061184 00000100:00100000:0.0:1789760720.769724:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 853 00000100:00100000:0.0:1789760720.769730:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3472:x1876699534061056:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.769773:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3472:x1876699534061056:12345-0@lo:400:kworker.0 Request processed in 43us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760720.769780:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 853 00000100:00100000:0.0:1789760720.769793:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 854 00000100:00100000:0.0:1789760720.769798:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534060672:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.769824:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534060672:12345-0@lo:400:kworker.0 Request processed in 26us (272us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760720.769829:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 854 00000100:00100000:0.0:1789760720.769839:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 855 00000100:00100000:0.0:1789760720.769844:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3472:x1876699534061184:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.769867:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3472:x1876699534061184:12345-0@lo:400:kworker.0 Request processed in 23us (264us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760720.769872:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 855 00000100:00100000:0.0:1789760720.769915:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b645500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534060672:0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.769925:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b644e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3472:1876699534061184:0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.769934:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3472:1876699534061056:0@lo:400:kworker.0 00000100:00100000:2.0:1789760720.771102:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3474:1876699534060288:0@lo:400:kworker.0 00000100:00100000:2.0:1789760720.771154:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760720.771195:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534060800:0@lo:400:kworker.0 00000100:00100000:2.0:1789760720.771220:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760720.771251:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b647100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534061312:0@lo:400:kworker.0 00000100:00100000:2.0:1789760720.771271:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760720.771315:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789760720.771321:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b647800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534060416:0@lo:400:kworker.0 00000100:00100000:2.0:1789760720.771344:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760720.771359:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b645880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534060928:0@lo:400:kworker.0 00000100:00100000:2.0:1789760720.771377:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760720.771560:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534061312 00000100:00100000:1.0:1789760720.771568:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534061312 00000100:00100000:1.0:1789760720.771609:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534060416 00000100:00100000:1.0:1789760720.771612:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534060416 00000100:00100000:1.0:1789760720.771628:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534060928 00000100:00100000:1.0:1789760720.771631:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534060928 00000100:00100000:1.0:1789760720.771646:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 856 00000100:00100000:1.0:1789760720.771654:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886360700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534061312:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760720.771703:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886360700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534061312:12345-0@lo:400:kworker.0 Request processed in 50us (430us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760720.771711:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 856 00000100:00100000:1.0:1789760720.771729:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 857 00000100:00100000:1.0:1789760720.771734:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886362300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534060416:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760720.771759:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886362300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534060416:12345-0@lo:400:kworker.0 Request processed in 25us (414us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760720.771765:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 857 00000100:00100000:1.0:1789760720.771777:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 858 00000100:00100000:1.0:1789760720.771783:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886360380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534060928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760720.771807:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886360380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534060928:12345-0@lo:400:kworker.0 Request processed in 24us (429us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760720.771812:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 858 00000100:00080000:1.0:1789760720.771834:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760720.771838:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789760720.771875:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b647800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534060416:0@lo:400:kworker.0 00000100:00100000:1.0:1789760720.771885:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b645880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534060928:0@lo:400:kworker.0 00000100:00100000:1.0:1789760720.771894:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b647100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534061312:0@lo:400:kworker.0 00000100:00100000:1.0:1789760720.775815:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534060800:0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.776159:0:6460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534060288 00000100:00100000:0.0:1789760720.776167:0:6460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534060288 00000100:00100000:0.0:1789760720.776215:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:0.0:1789760720.776224:0:6460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04989032850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3474:x1876699534060288:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760720.776350:0:6460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04989032850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3474:x1876699534060288:12345-0@lo:400:kworker.0 Request processed in 125us (5194us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760720.776357:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00100000:0.0:1789760720.776431:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b645c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534060544:0@lo:400:kworker.0 00000100:00100000:1.0:1789760720.776667:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3474:1876699534060288:0@lo:400:kworker.0 00002000:00010000:1.0:1789760720.792251:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19292160 -> 19296256 00010000:00010000:1.0:1789760720.792266:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe547d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.792279:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe547d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.792309:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe547d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.792317:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760720.792334:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe547d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.792350:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe547d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.792370:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe547d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.792377:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe547d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19292160->19296255] (req 19292160->19296255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760720.792470:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508281600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47597us (48065us total) trans 4294972030 rc 0/0 00000100:00100000:1.0:1789760720.792480:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4742 00000100:00100000:1.0:1789760720.792491:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699817226691 time 699849570084 state 0x2 ref 0 00000100:00100000:1.0:1789760720.799130:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760720.854466:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508281856 00000100:00100000:2.0:1789760720.854477:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508281856 00000100:00100000:2.0:1789760720.854531:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699849830500 time 699904287059 state 0x0 ref 1 00000100:00100000:2.0:1789760720.854546:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699904547475 time 699904287059 state 0x1 ref 1 00000100:00100000:2.0:1789760720.854575:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699904547475 time 699959285873 state 0x0 ref 1 00000100:00100000:2.0:1789760720.854727:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508281856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760720.854775:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19300352, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9807 00010000:00010000:2.0:1789760720.854784:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe547e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.854799:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=699s v=(0 0) 00010000:00010000:2.0:1789760720.854804:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760720.854810:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe547e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.854824:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe547e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760720.920857:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972032 is committed 00002000:00010000:2.0:1789760720.921030:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19300352 -> 19304448 00010000:00010000:2.0:1789760720.921046:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe547e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.921060:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe547e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.921070:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe547e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.921078:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760720.921083:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe547e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.921093:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe547e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.921116:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe547e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760720.921124:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe547e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19300352->19304447] (req 19300352->19304447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760720.921217:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508281856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 66489us (67077us total) trans 4294972032 rc 0/0 00000100:00100000:2.0:1789760720.921238:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4744 00000100:00100000:2.0:1789760720.921248:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699904547475 time 699959285873 state 0x2 ref 0 00000100:00100000:2.0:1789760720.927738:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760720.927895:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508281984 00000100:00100000:1.0:1789760720.927905:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508281984 00000100:00100000:1.0:1789760720.927968:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699904547475 time 699959285873 state 0x0 ref 1 00000100:00100000:1.0:1789760720.927984:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699959546289 time 699959285873 state 0x1 ref 1 00000100:00100000:1.0:1789760720.928012:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699959546289 time 700032722884 state 0x0 ref 1 00000100:00100000:1.0:1789760720.928025:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508281984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760720.928071:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19304448, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9807 00010000:00010000:1.0:1789760720.928080:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe547ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.928094:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:1.0:1789760720.928098:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760720.928104:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe547ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760720.928118:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe547ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.018560:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972033 is committed 00002000:00010000:1.0:1789760721.018644:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19304448 -> 19308544 00010000:00010000:1.0:1789760721.018663:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe547ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.018682:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe547ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.018691:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe547ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.018698:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760721.018703:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe547ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.018712:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe547ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.018736:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe547ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.018744:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe547ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19304448->19308543] (req 19304448->19308543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.019005:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508281984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 90977us (91274us total) trans 4294972033 rc 0/0 00000100:00100000:1.0:1789760721.019018:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4745 00000100:00100000:1.0:1789760721.019028:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 699959546289 time 700032722884 state 0x2 ref 0 00000100:00100000:1.0:1789760721.027180:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00080000:1.0:1789760721.027673:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760721.140882:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972034 is committed 00002000:00010000:1.0:1789760721.141040:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19308544 -> 19312640 00010000:00010000:1.0:1789760721.141067:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe547f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.141082:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe547f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.141092:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe547f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.141100:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760721.141105:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe547f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.141115:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe547f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.141139:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe547f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.141147:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe547f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19308544->19312639] (req 19308544->19312639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.141277:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508282112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 113747us (114105us total) trans 4294972034 rc 0/0 00000100:00100000:1.0:1789760721.141345:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4746 00000100:00100000:1.0:1789760721.141359:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700032983300 time 700132221381 state 0x2 ref 0 00000100:00100000:2.0:1789760721.148089:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508282240 00000100:00100000:2.0:1789760721.148105:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508282240 00000100:00100000:2.0:1789760721.148182:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700032983300 time 700132221381 state 0x0 ref 1 00000100:00100000:2.0:1789760721.148202:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700132481797 time 700132221381 state 0x1 ref 1 00000100:00100000:2.0:1789760721.148249:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700132481797 time 700252959366 state 0x0 ref 1 00000100:00100000:2.0:1789760721.148263:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508282240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760721.148365:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19312640, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9807 00010000:00010000:2.0:1789760721.148376:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe547fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.148397:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760721.148402:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.148410:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe547fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.148428:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe547fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760721.189426:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19312640 -> 19316736 00010000:00010000:2.0:1789760721.189453:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe547fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.189469:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe547fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.189480:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe547fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.189489:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760721.189495:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe547fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.189506:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe547fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.189533:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe547fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.189541:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe547fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19312640->19316735] (req 19312640->19316735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.189670:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508282240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41399us (41683us total) trans 4294972035 rc 0/0 00000100:00100000:2.0:1789760721.189682:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4747 00000100:00100000:2.0:1789760721.189695:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700132481797 time 700252959366 state 0x2 ref 0 00000100:00100000:2.0:1789760721.194756:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760721.194945:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508282368 00000100:00100000:1.0:1789760721.194958:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508282368 00000100:00100000:1.0:1789760721.195031:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700132481797 time 700252959366 state 0x0 ref 1 00000100:00100000:1.0:1789760721.195050:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700253219782 time 700252959366 state 0x1 ref 1 00000100:00100000:1.0:1789760721.195090:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700253219782 time 700299800323 state 0x0 ref 1 00000100:00100000:1.0:1789760721.195103:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508282368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760721.195181:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19316736, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9807 00010000:00010000:1.0:1789760721.195203:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54803 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.195236:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:1.0:1789760721.195241:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.195250:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54803 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.195269:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54803 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760721.263416:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19316736 -> 19320832 00010000:00010000:1.0:1789760721.263452:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54803 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.263477:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54803 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.263487:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54803 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.263495:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760721.263500:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54803 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.263510:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54803 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.263536:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54803 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.263544:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54803 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19316736->19320831] (req 19316736->19320831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.263690:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508282368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 68580us (68943us total) trans 4294972036 rc 0/0 00000100:00100000:1.0:1789760721.263712:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4748 00000100:00100000:1.0:1789760721.263733:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700253219782 time 700299800323 state 0x2 ref 0 00000100:00100000:0.0:1789760721.266934:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760721.267071:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508282496 00000100:00100000:1.0:1789760721.267085:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508282496 00000100:00100000:1.0:1789760721.267167:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700253219782 time 700299800323 state 0x0 ref 1 00000100:00100000:1.0:1789760721.267188:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700300060739 time 700299800323 state 0x1 ref 1 00000100:00100000:1.0:1789760721.267219:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700300060739 time 700371928940 state 0x0 ref 1 00000100:00100000:1.0:1789760721.267240:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508282496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760721.267347:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19320832, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9807 00010000:00010000:1.0:1789760721.267359:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5480a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.267378:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:1.0:1789760721.267383:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.267390:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5480a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.267407:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5480a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760721.336506:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19320832 -> 19324928 00010000:00010000:1.0:1789760721.336527:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5480a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.336542:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5480a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.336559:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5480a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.336568:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760721.336678:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5480a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.336691:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5480a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.336718:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5480a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.336728:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5480a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19320832->19324927] (req 19320832->19324927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.337568:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508282496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 70318us (70640us total) trans 4294972037 rc 0/0 00000100:00100000:1.0:1789760721.337594:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4749 00000100:00100000:1.0:1789760721.337607:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700300060739 time 700371928940 state 0x2 ref 0 00000100:00100000:2.0:1789760721.344551:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760721.344684:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508282624 00000100:00100000:1.0:1789760721.344707:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508282624 00000100:00100000:1.0:1789760721.344779:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700300060739 time 700371928940 state 0x0 ref 1 00000100:00100000:1.0:1789760721.344798:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700372189356 time 700371928940 state 0x1 ref 1 00000100:00100000:1.0:1789760721.344833:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700372189356 time 700449543053 state 0x0 ref 1 00000100:00100000:1.0:1789760721.344847:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508282624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760721.344913:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19324928, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9807 00010000:00010000:1.0:1789760721.344924:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54811 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.344941:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:1.0:1789760721.344947:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.344955:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54811 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.344970:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54811 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19324928->19329023] (req 19324928->19329023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760721.394306:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972038 is committed 00000100:00100000:2.0:1789760721.406771:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760721.447241:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19329024 -> 19333120 00010000:00010000:0.0:1789760721.447271:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54818 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.447309:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54818 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.447319:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54818 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.447326:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760721.447332:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54818 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.447342:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54818 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.447367:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54818 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.447375:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54818 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19329024->19333119] (req 19329024->19333119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760721.447512:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508282752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40478us (40748us total) trans 4294972039 rc 0/0 00000100:00100000:0.0:1789760721.447524:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4751 00000100:00100000:0.0:1789760721.447537:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700449803469 time 700511725434 state 0x2 ref 0 00000001:00080000:2.0:1789760721.448203:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972039 is committed 00000100:00100000:2.0:1789760721.451322:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508282880 00000100:00100000:2.0:1789760721.451337:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508282880 00000100:00100000:2.0:1789760721.451413:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700449803469 time 700511725434 state 0x0 ref 1 00000100:00100000:2.0:1789760721.451433:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700511985850 time 700511725434 state 0x1 ref 1 00000100:00100000:2.0:1789760721.451476:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700511985850 time 700556185820 state 0x0 ref 1 00000100:00100000:2.0:1789760721.451491:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508282880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760721.451556:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19333120, mtime: 0x6aad94d1, atime: 0x0, ctime: 0x6aad94d1, blocks: 0x9807 00010000:00010000:2.0:1789760721.451566:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5481f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.451586:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760721.451591:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.451599:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5481f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.451617:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5481f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760721.548098:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19333120 -> 19337216 00010000:00010000:1.0:1789760721.548124:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5481f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.548153:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5481f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.548163:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5481f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.548173:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760721.548180:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5481f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.548191:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5481f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.548217:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5481f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.548234:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5481f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19333120->19337215] (req 19333120->19337215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.548422:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508282880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 96927us (97448us total) trans 4294972040 rc 0/0 00000100:00100000:1.0:1789760721.548434:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4752 00000100:00100000:1.0:1789760721.548446:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700511985850 time 700556185820 state 0x2 ref 0 00000100:00100000:0.0:1789760721.554471:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760721.655858:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972041 is committed 00002000:00010000:1.0:1789760721.658405:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19337216 -> 19341312 00002000:00010000:1.0:1789760721.658417:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760721 -> 1789760722 00002000:00010000:1.0:1789760721.658420:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760721 -> 1789760722 00010000:00010000:1.0:1789760721.658439:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54826 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.658457:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54826 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.658467:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54826 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.658477:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760721.658482:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54826 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.658493:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54826 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.658519:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54826 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.658527:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54826 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19337216->19341311] (req 19337216->19341311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.658681:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508283008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 103798us (104219us total) trans 4294972041 rc 0/0 00000100:00100000:1.0:1789760721.658696:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4753 00000100:00100000:1.0:1789760721.658710:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700556446236 time 700659573313 state 0x2 ref 0 00010000:00010000:0.0:1789760721.663837:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760721.665476:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760721.665589:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508283136 00000100:00100000:0.0:1789760721.665602:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508283136 00000100:00100000:0.0:1789760721.665681:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700556446236 time 700659573313 state 0x0 ref 1 00000100:00100000:0.0:1789760721.665702:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700659833729 time 700659573313 state 0x1 ref 1 00000100:00100000:0.0:1789760721.665739:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700659833729 time 700770448440 state 0x0 ref 1 00000100:00100000:0.0:1789760721.665753:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508283136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760721.665830:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19341312, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:0.0:1789760721.665841:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5482d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.665862:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:0.0:1789760721.665868:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760721.665876:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5482d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760721.665893:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5482d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.718014:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972042 is committed 00002000:00010000:1.0:1789760721.718099:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19341312 -> 19345408 00010000:00010000:1.0:1789760721.718115:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5482d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.718128:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5482d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.718143:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5482d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.718150:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760721.718155:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5482d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.718164:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5482d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.718184:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5482d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.718192:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5482d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19341312->19345407] (req 19341312->19345407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.718344:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508283136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52589us (52877us total) trans 4294972042 rc 0/0 00000100:00100000:1.0:1789760721.718354:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4754 00000100:00100000:1.0:1789760721.718365:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700659833729 time 700770448440 state 0x2 ref 0 00000100:00100000:1.0:1789760721.723910:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508283264 00000100:00100000:1.0:1789760721.723923:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508283264 00000100:00100000:1.0:1789760721.723982:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700659833729 time 700770448440 state 0x0 ref 1 00000100:00100000:1.0:1789760721.723997:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700770708856 time 700770448440 state 0x1 ref 1 00000100:00100000:1.0:1789760721.724019:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700770708856 time 700828730123 state 0x0 ref 1 00000100:00100000:1.0:1789760721.724031:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508283264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760721.724076:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19345408, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:1.0:1789760721.724085:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54834 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.724099:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:1.0:1789760721.724103:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.724109:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54834 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.724122:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54834 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760721.768642:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972043 is committed 00002000:00010000:2.0:1789760721.769852:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19345408 -> 19349504 00010000:00010000:2.0:1789760721.769872:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54834 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.769886:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54834 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.769895:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54834 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.769903:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760721.769908:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54834 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.769917:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54834 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.769939:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54834 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.769948:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54834 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19345408->19349503] (req 19345408->19349503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760721.770087:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508283264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46052us (47309us total) trans 4294972043 rc 0/0 00000100:00100000:2.0:1789760721.770282:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4755 00000100:00100000:2.0:1789760721.770328:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700770708856 time 700828730123 state 0x2 ref 0 00000100:00100000:0.0:1789760721.773893:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760721.773986:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508283392 00000100:00100000:1.0:1789760721.773995:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508283392 00000100:00100000:1.0:1789760721.774046:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700770708856 time 700828730123 state 0x0 ref 1 00000100:00100000:1.0:1789760721.774060:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700828990539 time 700828730123 state 0x1 ref 1 00000100:00100000:1.0:1789760721.774089:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700828990539 time 700878800047 state 0x0 ref 1 00000100:00100000:1.0:1789760721.774257:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508283392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760721.774403:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19349504, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:1.0:1789760721.774412:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5483b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.774426:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:1.0:1789760721.774430:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.774436:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5483b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.774450:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5483b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.791115:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760721.791148:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c9c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534061440:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789760721.791216:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760721.791500:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534061440 00000100:00100000:1.0:1789760721.791507:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534061440 00000100:00100000:1.0:1789760721.791561:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 416 00000100:00100000:1.0:1789760721.791575:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534061440:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760721.791926:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760721.791976:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534061440:12345-0@lo:41:osp-pre-0-1.0 Request processed in 407us (759us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760721.791984:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 416 00000100:00100000:1.0:1789760721.792032:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c9c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534061440:0@lo:41:osp-pre-0-1.0 00002000:00010000:1.0:1789760721.817412:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19349504 -> 19353600 00010000:00010000:1.0:1789760721.817431:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5483b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.817450:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5483b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.817459:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5483b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.817467:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760721.817471:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5483b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.817479:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5483b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.817508:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5483b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.817516:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5483b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19349504->19353599] (req 19349504->19353599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.817881:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508283392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43776us (43992us total) trans 4294972044 rc 0/0 00000100:00100000:1.0:1789760721.817892:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4756 00000100:00100000:1.0:1789760721.817902:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700828990539 time 700878800047 state 0x2 ref 0 00000100:00100000:2.0:1789760721.833196:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760721.833309:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508283520 00000100:00100000:1.0:1789760721.833320:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508283520 00000100:00100000:1.0:1789760721.833409:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700828990539 time 700878800047 state 0x0 ref 1 00000100:00100000:1.0:1789760721.833424:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700879060463 time 700878800047 state 0x1 ref 1 00000100:00100000:1.0:1789760721.833445:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700879060463 time 700938155806 state 0x0 ref 1 00000100:00100000:1.0:1789760721.833457:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508283520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760721.833524:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19353600, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:1.0:1789760721.833551:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54842 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.833568:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:1.0:1789760721.833572:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.833578:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54842 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.833593:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54842 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760721.868767:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19353600 -> 19357696 00010000:00010000:1.0:1789760721.868786:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54842 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.868799:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54842 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.868813:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54842 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.868821:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760721.868825:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54842 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.868834:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54842 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.868852:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54842 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.868861:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54842 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19353600->19357695] (req 19353600->19357695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.868949:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508283520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35490us (35758us total) trans 4294972045 rc 0/0 00000100:00100000:1.0:1789760721.868959:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4757 00000100:00100000:1.0:1789760721.868969:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700879060463 time 700938155806 state 0x2 ref 0 00000100:00100000:0.0:1789760721.882466:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760721.882576:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508283648 00000100:00100000:2.0:1789760721.882587:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508283648 00000100:00100000:2.0:1789760721.882714:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700879060463 time 700938155806 state 0x0 ref 1 00000100:00100000:2.0:1789760721.882729:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700938416222 time 700938155806 state 0x1 ref 1 00000100:00100000:2.0:1789760721.882804:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700938416222 time 700987514455 state 0x0 ref 1 00000100:00100000:2.0:1789760721.882817:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508283648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760721.882866:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19357696, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:2.0:1789760721.882874:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54849 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.882890:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=700s v=(0 0) 00010000:00010000:2.0:1789760721.882894:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760721.882900:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54849 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760721.882913:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54849 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19357696->19361791] (req 19357696->19361791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760721.932710:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972046 is committed 00000100:00100000:0.0:1789760721.942003:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760721.942232:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508283776 00000100:00100000:1.0:1789760721.942242:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508283776 00000100:00100000:1.0:1789760721.942413:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700938416222 time 700987514455 state 0x0 ref 1 00000100:00100000:1.0:1789760721.942432:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700987774871 time 700987514455 state 0x1 ref 1 00000100:00100000:1.0:1789760721.942493:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700987774871 time 701047202732 state 0x0 ref 1 00000100:00100000:1.0:1789760721.942506:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d47450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508283776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760721.942564:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19361792, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:1.0:1789760721.942578:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54850 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.942593:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760721.942702:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.942710:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54850 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.942725:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54850 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760721.979027:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19361792 -> 19365888 00010000:00010000:1.0:1789760721.979046:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54850 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.979061:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54850 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.979070:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54850 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.979084:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760721.979089:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54850 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.979098:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54850 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.979128:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54850 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.979143:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54850 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19361792->19365887] (req 19361792->19365887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760721.979253:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d47450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508283776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36742us (37255us total) trans 4294972047 rc 0/0 00000100:00100000:1.0:1789760721.979264:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4759 00000100:00100000:1.0:1789760721.979273:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700987774871 time 701047202732 state 0x2 ref 0 00000100:00100000:2.0:1789760721.985546:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760721.985986:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508283904 00000100:00100000:1.0:1789760721.985996:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508283904 00000100:00100000:1.0:1789760721.986051:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 700987774871 time 701047202732 state 0x0 ref 1 00000100:00100000:1.0:1789760721.986067:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701047463148 time 701047202732 state 0x1 ref 1 00000100:00100000:1.0:1789760721.986099:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701047463148 time 701090808802 state 0x0 ref 1 00000100:00100000:1.0:1789760721.986123:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508283904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760721.986178:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19365888, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:1.0:1789760721.986187:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54857 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.986203:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760721.986207:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760721.986214:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54857 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760721.986239:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54857 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760722.035750:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19365888 -> 19369984 00010000:00010000:0.0:1789760722.035767:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54857 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.035780:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54857 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.035790:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54857 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.035797:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760722.035803:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54857 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.035817:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54857 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.035838:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54857 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.035847:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54857 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19365888->19369983] (req 19365888->19369983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.035985:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508283904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49860us (50454us total) trans 4294972048 rc 0/0 00000100:00100000:0.0:1789760722.035996:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4760 00000100:00100000:0.0:1789760722.036006:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701047463148 time 701090808802 state 0x2 ref 0 00000100:00100000:0.0:1789760722.040630:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:1.0:1789760722.100265:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972049 is committed 00002000:00010000:0.0:1789760722.101616:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19369984 -> 19374080 00010000:00010000:0.0:1789760722.101635:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5485e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.101648:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5485e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.101657:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5485e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.101669:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760722.101674:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5485e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.101682:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5485e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.101701:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5485e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.101709:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5485e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19369984->19374079] (req 19369984->19374079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.101819:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508284032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60998us (61194us total) trans 4294972049 rc 0/0 00000100:00100000:0.0:1789760722.101830:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4761 00000100:00100000:0.0:1789760722.101840:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701091069218 time 701145517278 state 0x2 ref 0 00000100:00100000:1.0:1789760722.106672:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508284160 00000100:00100000:1.0:1789760722.106682:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508284160 00000100:00100000:1.0:1789760722.106734:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701091069218 time 701145517278 state 0x0 ref 1 00000100:00100000:1.0:1789760722.106749:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701145777694 time 701145517278 state 0x1 ref 1 00000100:00100000:1.0:1789760722.106779:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701145777694 time 701211489233 state 0x0 ref 1 00000100:00100000:1.0:1789760722.106791:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508284160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760722.106839:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19374080, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:1.0:1789760722.106849:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54865 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.106864:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.106868:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.106873:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54865 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.106886:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54865 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19374080->19378175] (req 19374080->19378175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.155132:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972050 is committed 00000100:00100000:0.0:1789760722.155506:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508284160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48713us (49018us total) trans 4294972050 rc 0/0 00000100:00100000:0.0:1789760722.155514:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4762 00000100:00100000:0.0:1789760722.155522:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701145777694 time 701211489233 state 0x2 ref 0 00000100:00100000:0.0:1789760722.159555:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508284288 00000100:00100000:0.0:1789760722.159565:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508284288 00000100:00100000:0.0:1789760722.160198:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701145777694 time 701211489233 state 0x0 ref 1 00000100:00100000:0.0:1789760722.160214:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701211749649 time 701211489233 state 0x1 ref 1 00000100:00100000:0.0:1789760722.160322:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701211749649 time 701265031598 state 0x0 ref 1 00000100:00100000:0.0:1789760722.160337:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508284288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760722.160386:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19378176, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:0.0:1789760722.160395:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5486c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.160420:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760722.160424:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.160431:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5486c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.160447:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5486c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760722.180131:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19378176 -> 19382272 00010000:00010000:0.0:1789760722.180151:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5486c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.180161:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5486c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.180172:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5486c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.180176:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760722.180180:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5486c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.180186:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5486c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.180202:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5486c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.180206:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5486c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19378176->19382271] (req 19378176->19382271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.180373:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508284288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20034us (21325us total) trans 4294972051 rc 0/0 00000100:00100000:0.0:1789760722.180384:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4763 00000100:00100000:0.0:1789760722.180392:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701211749649 time 701265031598 state 0x2 ref 0 00000001:00080000:0.0:1789760722.180501:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972051 is committed 00000100:00100000:2.0:1789760722.183580:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760722.183687:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508284416 00000100:00100000:0.0:1789760722.183696:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508284416 00000100:00100000:0.0:1789760722.183824:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701211749649 time 701265031598 state 0x0 ref 1 00000100:00100000:0.0:1789760722.183842:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701265292014 time 701265031598 state 0x1 ref 1 00000100:00100000:0.0:1789760722.183882:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701265292014 time 701288583378 state 0x0 ref 1 00000100:00100000:0.0:1789760722.183895:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508284416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760722.183944:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19382272, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:0.0:1789760722.183952:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54873 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.183968:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760722.183972:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.183978:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54873 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.183990:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54873 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760722.201853:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19382272 -> 19386368 00010000:00010000:0.0:1789760722.201872:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54873 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.201885:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54873 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.201894:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54873 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.201907:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760722.201912:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54873 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.201923:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54873 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.201942:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54873 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.201949:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54873 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19382272->19386367] (req 19382272->19386367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.202039:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508284416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 18141us (18463us total) trans 4294972052 rc 0/0 00000100:00100000:0.0:1789760722.202050:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4764 00000100:00100000:0.0:1789760722.202060:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701265292014 time 701288583378 state 0x2 ref 0 00000001:00080000:0.0:1789760722.202098:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972052 is committed 00000100:00100000:0.0:1789760722.205436:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508284544 00000100:00100000:0.0:1789760722.205446:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508284544 00000100:00100000:0.0:1789760722.205496:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701265292014 time 701288583378 state 0x0 ref 1 00000100:00100000:0.0:1789760722.205511:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701288843794 time 701288583378 state 0x1 ref 1 00000100:00100000:0.0:1789760722.205582:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701288843794 time 701310291706 state 0x0 ref 1 00000100:00100000:0.0:1789760722.205596:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508284544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760722.205641:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19386368, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:0.0:1789760722.205650:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5487a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.205665:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760722.205670:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.205676:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5487a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.205689:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5487a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760722.232554:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19386368 -> 19390464 00010000:00010000:1.0:1789760722.232634:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5487a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.232648:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5487a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.232657:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5487a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.232664:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760722.232669:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5487a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.232678:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5487a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.232697:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5487a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.232705:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5487a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19386368->19390463] (req 19386368->19390463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.232796:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508284544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27197us (27923us total) trans 4294972053 rc 0/0 00000100:00100000:1.0:1789760722.232807:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4765 00000100:00100000:1.0:1789760722.232818:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701288843794 time 701310291706 state 0x2 ref 0 00000100:00100000:0.0:1789760722.235808:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508284672 00000100:00100000:0.0:1789760722.235817:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508284672 00000100:00100000:0.0:1789760722.235869:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701288843794 time 701310291706 state 0x0 ref 1 00000100:00100000:0.0:1789760722.235885:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701310552122 time 701310291706 state 0x1 ref 1 00000100:00100000:0.0:1789760722.235910:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701310552122 time 701340620260 state 0x0 ref 1 00000100:00100000:0.0:1789760722.235923:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508284672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760722.235973:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19390464, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:0.0:1789760722.235984:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54881 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.236006:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760722.236011:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.236018:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54881 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.236032:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54881 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.266468:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972054 is committed 00002000:00010000:1.0:1789760722.266576:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19390464 -> 19394560 00010000:00010000:1.0:1789760722.266593:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54881 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.266606:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54881 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.266616:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54881 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.266624:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760722.266630:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54881 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.266639:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54881 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.266665:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54881 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.266673:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54881 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19390464->19394559] (req 19390464->19394559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.266772:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508284672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30844us (31165us total) trans 4294972054 rc 0/0 00000100:00100000:1.0:1789760722.266783:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4766 00000100:00100000:1.0:1789760722.266793:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701310552122 time 701340620260 state 0x2 ref 0 00000100:00100000:2.0:1789760722.270079:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:0.0:1789760722.305099:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19394560 -> 19398656 00010000:00010000:0.0:1789760722.305120:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54888 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.305134:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54888 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.305143:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54888 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.305150:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760722.305154:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54888 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.305163:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54888 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.305182:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54888 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.305190:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54888 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19394560->19398655] (req 19394560->19398655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760722.305324:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508284800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34924us (35254us total) trans 4294972055 rc 0/0 00000100:00100000:0.0:1789760722.305334:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4767 00000100:00100000:0.0:1789760722.305344:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701340880676 time 701375094936 state 0x2 ref 0 00000001:00080000:1.0:1789760722.306922:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972055 is committed 00000100:00100000:2.0:1789760722.315179:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760722.315268:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508284928 00000100:00100000:1.0:1789760722.315278:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508284928 00000100:00100000:1.0:1789760722.315349:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701340880676 time 701375094936 state 0x0 ref 1 00000100:00100000:1.0:1789760722.315364:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701375355352 time 701375094936 state 0x1 ref 1 00000100:00100000:1.0:1789760722.315391:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701375355352 time 701420100997 state 0x0 ref 1 00000100:00100000:1.0:1789760722.315405:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508284928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760722.315454:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19398656, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:1.0:1789760722.315464:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5488f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.315486:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.315490:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.315497:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5488f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.315510:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5488f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.373642:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972056 is committed 00002000:00010000:1.0:1789760722.373789:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19398656 -> 19402752 00010000:00010000:1.0:1789760722.373805:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5488f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.373818:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5488f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.373827:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5488f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.373834:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760722.373839:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5488f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.373848:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5488f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.373867:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5488f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.374006:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5488f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19398656->19402751] (req 19398656->19402751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.374135:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508284928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58722us (58952us total) trans 4294972056 rc 0/0 00000100:00100000:1.0:1789760722.374146:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4768 00000100:00100000:1.0:1789760722.374155:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701375355352 time 701420100997 state 0x2 ref 0 00000100:00100000:2.0:1789760722.382033:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760722.382121:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508285056 00000100:00100000:1.0:1789760722.382200:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508285056 00000100:00100000:1.0:1789760722.382266:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701375355352 time 701420100997 state 0x0 ref 1 00000100:00100000:1.0:1789760722.382281:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701420361413 time 701420100997 state 0x1 ref 1 00000100:00100000:1.0:1789760722.382455:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701420361413 time 701487164717 state 0x0 ref 1 00000100:00100000:1.0:1789760722.382470:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508285056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760722.382516:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19402752, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:1.0:1789760722.382525:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54896 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.382540:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.382544:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.382551:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54896 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.382670:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54896 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760722.409608:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19402752 -> 19406848 00010000:00010000:2.0:1789760722.409629:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54896 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.409644:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54896 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.409653:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54896 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.409660:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760722.409664:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54896 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.409672:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54896 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.409691:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54896 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.409699:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe54896 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19402752->19406847] (req 19402752->19406847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.409794:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508285056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27322us (27765us total) trans 4294972057 rc 0/0 00000100:00100000:2.0:1789760722.409804:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4769 00000100:00100000:2.0:1789760722.409814:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701420361413 time 701487164717 state 0x2 ref 0 00000001:00080000:0.0:1789760722.410482:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972057 is committed 00000100:00100000:2.0:1789760722.413604:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760722.413924:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508285184 00000100:00100000:0.0:1789760722.413934:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508285184 00000100:00100000:0.0:1789760722.413985:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701420361413 time 701487164717 state 0x0 ref 1 00000100:00100000:0.0:1789760722.413999:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701487425133 time 701487164717 state 0x1 ref 1 00000100:00100000:0.0:1789760722.414028:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701487425133 time 701518738570 state 0x0 ref 1 00000100:00100000:0.0:1789760722.414041:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508285184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760722.414089:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19406848, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:0.0:1789760722.414098:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5489d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.414113:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760722.414117:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.414125:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5489d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.414139:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5489d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.435526:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972058 is committed 00002000:00010000:1.0:1789760722.435764:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19406848 -> 19410944 00010000:00010000:1.0:1789760722.435779:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5489d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.435793:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5489d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.435802:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5489d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.435810:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760722.435815:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5489d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.435824:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5489d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.435843:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5489d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.435851:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5489d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19406848->19410943] (req 19406848->19410943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.435981:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508285184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 21936us (22382us total) trans 4294972058 rc 0/0 00000100:00100000:1.0:1789760722.435992:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4770 00000100:00100000:1.0:1789760722.436001:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701487425133 time 701518738570 state 0x2 ref 0 00000100:00100000:1.0:1789760722.439347:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760722.439453:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508285312 00000100:00100000:0.0:1789760722.439463:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508285312 00000100:00100000:0.0:1789760722.439523:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701487425133 time 701518738570 state 0x0 ref 1 00000100:00100000:0.0:1789760722.439539:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701518998986 time 701518738570 state 0x1 ref 1 00000100:00100000:0.0:1789760722.439662:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701518998986 time 701544371889 state 0x0 ref 1 00000100:00100000:0.0:1789760722.439676:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508285312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760722.439729:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19410944, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:0.0:1789760722.439737:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe548a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.439753:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:0.0:1789760722.439757:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760722.439764:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe548a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760722.439777:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe548a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760722.475793:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972059 is committed 00002000:00010000:1.0:1789760722.475894:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19410944 -> 19415040 00010000:00010000:1.0:1789760722.475910:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe548a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.475924:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe548a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.475933:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe548a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.475941:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760722.475946:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe548a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.475955:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe548a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.475975:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe548a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.475986:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe548a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19410944->19415039] (req 19410944->19415039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.476118:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508285312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36440us (36775us total) trans 4294972059 rc 0/0 00000100:00100000:1.0:1789760722.476140:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4771 00000100:00100000:1.0:1789760722.476151:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701518998986 time 701544371889 state 0x2 ref 0 00000100:00100000:2.0:1789760722.480231:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760722.487258:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508285440 00000100:00100000:1.0:1789760722.487273:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508285440 00000100:00100000:1.0:1789760722.487367:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701518998986 time 701544371889 state 0x0 ref 1 00000100:00100000:1.0:1789760722.487384:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701544632305 time 701544371889 state 0x1 ref 1 00000100:00100000:1.0:1789760722.487412:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701544632305 time 701592122689 state 0x0 ref 1 00000100:00100000:1.0:1789760722.487425:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508285440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760722.487475:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19415040, mtime: 0x6aad94d2, atime: 0x0, ctime: 0x6aad94d2, blocks: 0x9807 00010000:00010000:1.0:1789760722.487485:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.487501:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.487505:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.487511:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.487530:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.535588:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972060 is committed 00002000:00010000:2.0:1789760722.535598:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19415040 -> 19419136 00002000:00010000:2.0:1789760722.535602:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760722 -> 1789760723 00002000:00010000:2.0:1789760722.535604:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760722 -> 1789760723 00010000:00010000:2.0:1789760722.535616:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.535629:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.535638:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.535645:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760722.535650:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.535659:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.535679:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.535687:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19415040->19419135] (req 19415040->19419135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.536058:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508285440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48630us (55840us total) trans 4294972060 rc 0/0 00000100:00100000:2.0:1789760722.536068:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4772 00000100:00100000:2.0:1789760722.536077:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701544632305 time 701592122689 state 0x2 ref 0 00000100:00100000:2.0:1789760722.542436:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760722.571716:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972061 is committed 00002000:00010000:1.0:1789760722.571828:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19419136 -> 19423232 00010000:00010000:1.0:1789760722.571845:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe548b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.571859:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe548b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.571869:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe548b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.571876:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760722.571882:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe548b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.571891:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe548b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.571914:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe548b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.571923:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe548b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19419136->19423231] (req 19419136->19423231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.572029:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508285568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29075us (29599us total) trans 4294972061 rc 0/0 00000100:00100000:1.0:1789760722.572040:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4773 00000100:00100000:1.0:1789760722.572051:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701592383105 time 701647646323 state 0x2 ref 0 00000100:00100000:1.0:1789760722.575894:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508285696 00000100:00100000:1.0:1789760722.575905:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508285696 00000100:00100000:1.0:1789760722.575965:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701592383105 time 701647646323 state 0x0 ref 1 00000100:00100000:1.0:1789760722.575980:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701647906739 time 701647646323 state 0x1 ref 1 00000100:00100000:1.0:1789760722.576003:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701647906739 time 701680714018 state 0x0 ref 1 00000100:00100000:1.0:1789760722.576013:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508285696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760722.576060:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19423232, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:1.0:1789760722.576068:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe548b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.576083:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.576086:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.576092:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe548b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.576106:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe548b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760722.612388:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19423232 -> 19427328 00010000:00010000:1.0:1789760722.612409:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe548b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.612422:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe548b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.612431:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe548b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.612437:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760722.612441:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe548b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.612450:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe548b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.612470:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe548b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.612478:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe548b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19423232->19427327] (req 19423232->19427327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.612693:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508285696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36675us (37122us total) trans 4294972062 rc 0/0 00000100:00100000:1.0:1789760722.612707:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4774 00000100:00100000:1.0:1789760722.612717:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701647906739 time 701680714018 state 0x2 ref 0 00000001:00080000:0.0:1789760722.616140:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972062 is committed 00000100:00100000:2.0:1789760722.616798:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760722.616885:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508285824 00000100:00100000:1.0:1789760722.616894:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508285824 00000100:00100000:1.0:1789760722.616950:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701647906739 time 701680714018 state 0x0 ref 1 00000100:00100000:1.0:1789760722.616964:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701680974434 time 701680714018 state 0x1 ref 1 00000100:00100000:1.0:1789760722.617028:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701680974434 time 701721738378 state 0x0 ref 1 00000100:00100000:1.0:1789760722.617041:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508285824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760722.617088:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19427328, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:1.0:1789760722.617096:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.617113:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.617118:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.617123:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.617148:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.656958:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972063 is committed 00002000:00010000:1.0:1789760722.657075:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19427328 -> 19431424 00010000:00010000:1.0:1789760722.657092:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.657119:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.657130:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.657144:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760722.657148:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.657156:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.657175:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.657183:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19427328->19431423] (req 19427328->19431423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.657833:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508285824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40787us (41033us total) trans 4294972063 rc 0/0 00000100:00100000:1.0:1789760722.657845:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4775 00000100:00100000:1.0:1789760722.657856:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701680974434 time 701721738378 state 0x2 ref 0 00000100:00100000:2.0:1789760722.664546:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760722.664810:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508285952 00000100:00100000:1.0:1789760722.664821:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508285952 00000100:00100000:1.0:1789760722.664875:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701680974434 time 701721738378 state 0x0 ref 1 00000100:00100000:1.0:1789760722.664891:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701721998794 time 701721738378 state 0x1 ref 1 00000100:00100000:1.0:1789760722.664923:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701721998794 time 701769633372 state 0x0 ref 1 00000100:00100000:1.0:1789760722.664937:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508285952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760722.664987:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19431424, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:1.0:1789760722.664997:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.665012:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.665017:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.665024:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.665038:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.694237:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760722.715410:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19431424 -> 19435520 00010000:00010000:1.0:1789760722.715431:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.715444:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.715458:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.715465:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760722.715469:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.715478:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.715496:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.715503:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19431424->19435519] (req 19431424->19435519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.715614:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508285952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50674us (51072us total) trans 4294972064 rc 0/0 00000100:00100000:1.0:1789760722.715625:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4776 00000100:00100000:1.0:1789760722.715635:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701721998794 time 701769633372 state 0x2 ref 0 00000001:00080000:0.0:1789760722.716962:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972064 is committed 00000100:00100000:0.0:1789760722.723073:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760722.723169:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508286080 00000100:00100000:1.0:1789760722.723178:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508286080 00000100:00100000:1.0:1789760722.723246:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701721998794 time 701769633372 state 0x0 ref 1 00000100:00100000:1.0:1789760722.723261:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701769893788 time 701769633372 state 0x1 ref 1 00000100:00100000:1.0:1789760722.723322:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701769893788 time 701828032188 state 0x0 ref 1 00000100:00100000:1.0:1789760722.723333:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508286080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760722.723392:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19435520, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:1.0:1789760722.723401:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.723416:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.723421:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.723427:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.723441:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760722.773611:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19435520 -> 19439616 00010000:00010000:1.0:1789760722.773630:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.773643:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.773653:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.773660:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760722.773666:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.773675:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.773697:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.773706:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19435520->19439615] (req 19435520->19439615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.773931:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508286080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50593us (50863us total) trans 4294972065 rc 0/0 00000100:00100000:1.0:1789760722.773942:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4777 00000100:00100000:1.0:1789760722.773952:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701769893788 time 701828032188 state 0x2 ref 0 00000100:00100000:2.0:1789760722.780206:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760722.780309:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508286208 00000100:00100000:1.0:1789760722.780319:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508286208 00000100:00100000:1.0:1789760722.780372:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701769893788 time 701828032188 state 0x0 ref 1 00000100:00100000:1.0:1789760722.780386:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701828292604 time 701828032188 state 0x1 ref 1 00000100:00100000:1.0:1789760722.780415:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701828292604 time 701885125799 state 0x0 ref 1 00000100:00100000:1.0:1789760722.780428:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508286208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760722.780473:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19439616, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:1.0:1789760722.780483:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.780498:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:1.0:1789760722.780503:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.780509:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.780524:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760722.823842:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972066 is committed 00002000:00010000:1.0:1789760722.823959:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19439616 -> 19443712 00010000:00010000:1.0:1789760722.823974:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.823988:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.823997:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.824004:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760722.824009:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.824019:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.824044:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.824052:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19439616->19443711] (req 19439616->19443711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.824261:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508286208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43830us (44059us total) trans 4294972066 rc 0/0 00000100:00100000:1.0:1789760722.824421:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4778 00000100:00100000:1.0:1789760722.824433:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701828292604 time 701885125799 state 0x2 ref 0 00000100:00100000:1.0:1789760722.829488:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760722.829622:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508286336 00000100:00100000:2.0:1789760722.829631:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508286336 00000100:00100000:2.0:1789760722.829679:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701828292604 time 701885125799 state 0x0 ref 1 00000100:00100000:2.0:1789760722.829693:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701885386215 time 701885125799 state 0x1 ref 1 00000100:00100000:2.0:1789760722.829757:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701885386215 time 701934466407 state 0x0 ref 1 00000100:00100000:2.0:1789760722.829832:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508286336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760722.829880:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19443712, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:2.0:1789760722.829889:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe548dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.829905:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760722.829909:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.829916:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe548dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.829930:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe548dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19443712->19447807] (req 19443712->19447807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.879705:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760722.879791:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508286464 00000100:00100000:2.0:1789760722.879801:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508286464 00000100:00100000:2.0:1789760722.879854:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701885386215 time 701934466407 state 0x0 ref 1 00000100:00100000:2.0:1789760722.879870:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701934726823 time 701934466407 state 0x1 ref 1 00000100:00100000:2.0:1789760722.879902:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701934726823 time 701984612623 state 0x0 ref 1 00000100:00100000:2.0:1789760722.879913:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508286464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760722.879959:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19447808, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:2.0:1789760722.879967:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe548e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.879982:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=701s v=(0 0) 00010000:00010000:2.0:1789760722.879986:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.879992:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe548e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.880005:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe548e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760722.907838:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19447808 -> 19451904 00010000:00010000:1.0:1789760722.907854:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe548e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.907866:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe548e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.907875:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe548e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.907883:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760722.907887:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe548e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.907897:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe548e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.907919:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe548e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.907926:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe548e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19447808->19451903] (req 19447808->19451903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760722.908056:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508286464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28140us (28356us total) trans 4294972068 rc 0/0 00000100:00100000:1.0:1789760722.908066:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4780 00000100:00100000:1.0:1789760722.908076:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701934726823 time 701984612623 state 0x2 ref 0 00000100:00100000:2.0:1789760722.913852:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760722.913932:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508286592 00000100:00100000:1.0:1789760722.913942:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508286592 00000100:00100000:1.0:1789760722.913993:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701934726823 time 701984612623 state 0x0 ref 1 00000100:00100000:1.0:1789760722.914008:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701984873039 time 701984612623 state 0x1 ref 1 00000100:00100000:1.0:1789760722.914043:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701984873039 time 702018753352 state 0x0 ref 1 00000100:00100000:1.0:1789760722.914056:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508286592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760722.914102:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19451904, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:1.0:1789760722.914112:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.914135:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760722.914140:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760722.914146:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760722.914160:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760722.939745:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19451904 -> 19456000 00010000:00010000:2.0:1789760722.939761:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.939773:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.939783:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.939790:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760722.939795:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.939804:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.939823:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.939831:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe548ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19451904->19455999] (req 19451904->19455999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760722.939961:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508286592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25902us (26114us total) trans 4294972069 rc 0/0 00000100:00100000:2.0:1789760722.939972:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4781 00000100:00100000:2.0:1789760722.939982:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 701984873039 time 702018753352 state 0x2 ref 0 00000100:00100000:0.0:1789760722.943746:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:1.0:1789760722.979553:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972070 is committed 00000100:00100000:1.0:1789760722.984119:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760722.984377:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508286848 00000100:00100000:2.0:1789760722.984387:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508286848 00000100:00100000:2.0:1789760722.984439:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702019013768 time 702048701287 state 0x0 ref 1 00000100:00100000:2.0:1789760722.984453:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702048961703 time 702048701287 state 0x1 ref 1 00000100:00100000:2.0:1789760722.984483:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702048961703 time 702089193534 state 0x0 ref 1 00000100:00100000:2.0:1789760722.984495:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508286848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760722.984540:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19460096, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:2.0:1789760722.984548:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe548f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.984675:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760722.984681:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760722.984688:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe548f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760722.984703:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe548f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760723.015311:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19460096 -> 19464192 00010000:00010000:1.0:1789760723.015404:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe548f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.015418:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe548f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.015428:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe548f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.015435:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760723.015440:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe548f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.015448:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe548f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.015473:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe548f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.015480:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe548f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19460096->19464191] (req 19460096->19464191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.015669:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508286848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31169us (31554us total) trans 4294972071 rc 0/0 00000100:00100000:1.0:1789760723.015679:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4783 00000100:00100000:1.0:1789760723.015688:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702048961703 time 702089193534 state 0x2 ref 0 00000100:00100000:2.0:1789760723.019714:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760723.019872:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508286976 00000100:00100000:1.0:1789760723.019881:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508286976 00000100:00100000:1.0:1789760723.019988:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702048961703 time 702089193534 state 0x0 ref 1 00000100:00100000:1.0:1789760723.020003:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702089453950 time 702089193534 state 0x1 ref 1 00000100:00100000:1.0:1789760723.020034:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702089453950 time 702124744719 state 0x0 ref 1 00000100:00100000:1.0:1789760723.020047:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508286976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760723.020095:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19464192, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:1.0:1789760723.020104:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe548ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.020119:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760723.020743:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.020753:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe548ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.020769:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe548ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760723.021527:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789760723.059577:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19464192 -> 19468288 00010000:00010000:1.0:1789760723.059597:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe548ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.059609:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe548ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.059619:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe548ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.059626:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760723.059631:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe548ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.059640:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe548ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.059659:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe548ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.059665:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe548ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19464192->19468287] (req 19464192->19468287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.059755:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508286976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39706us (40047us total) trans 4294972072 rc 0/0 00000100:00100000:1.0:1789760723.059764:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4784 00000100:00100000:1.0:1789760723.059771:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702089453950 time 702124744719 state 0x2 ref 0 00000100:00100000:1.0:1789760723.063423:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00002000:00010000:2.0:1789760723.092349:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19468288 -> 19472384 00010000:00010000:2.0:1789760723.092367:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54906 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.092379:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54906 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.092387:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54906 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.092393:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.092397:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54906 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.092406:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54906 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.092424:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54906 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.092432:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54906 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19468288->19472383] (req 19468288->19472383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.092518:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508287104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28892us (29100us total) trans 4294972073 rc 0/0 00000100:00100000:2.0:1789760723.092528:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4785 00000100:00100000:2.0:1789760723.092538:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702125005135 time 702168322181 state 0x2 ref 0 00000001:00080000:0.0:1789760723.092740:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972073 is committed 00000100:00100000:2.0:1789760723.097437:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760723.097540:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508287232 00000100:00100000:1.0:1789760723.097549:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508287232 00000100:00100000:1.0:1789760723.097720:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702125005135 time 702168322181 state 0x0 ref 1 00000100:00100000:1.0:1789760723.097737:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702168582597 time 702168322181 state 0x1 ref 1 00000100:00100000:1.0:1789760723.097806:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702168582597 time 702202515168 state 0x0 ref 1 00000100:00100000:1.0:1789760723.097821:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508287232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760723.097870:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19472384, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:1.0:1789760723.097879:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe5490d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.097895:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:1.0:1789760723.097899:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760723.097905:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe5490d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.097919:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe5490d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760723.123963:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19472384 -> 19476480 00010000:00010000:2.0:1789760723.123977:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe5490d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.123989:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe5490d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.123997:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe5490d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.124004:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.124008:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe5490d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.124017:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe5490d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.124038:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe5490d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.124046:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe5490d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19472384->19476479] (req 19472384->19476479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.127477:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508287232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29646us (30038us total) trans 4294972074 rc 0/0 00000100:00100000:2.0:1789760723.127494:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4786 00000100:00100000:2.0:1789760723.127505:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702168582597 time 702202515168 state 0x2 ref 0 00000100:00100000:0.0:1789760723.143410:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760723.146979:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508287360 00000100:00100000:1.0:1789760723.146994:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508287360 00000100:00100000:2.0:1789760723.149067:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702168582597 time 702202515168 state 0x0 ref 1 00000100:00100000:2.0:1789760723.149087:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702202775584 time 702202515168 state 0x1 ref 1 00000100:00100000:2.0:1789760723.149247:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702202775584 time 702253955895 state 0x0 ref 1 00000100:00100000:2.0:1789760723.149263:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508287360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.149331:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19476480, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:2.0:1789760723.149341:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54914 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.149356:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.149362:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.149368:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54914 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.149383:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54914 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760723.175029:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19476480 -> 19480576 00010000:00010000:2.0:1789760723.175044:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54914 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.175061:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54914 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.175070:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54914 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.175077:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.175082:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54914 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.175091:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54914 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.175111:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54914 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.175118:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54914 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19476480->19480575] (req 19476480->19480575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.175276:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508287360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26013us (31871us total) trans 4294972075 rc 0/0 00000100:00100000:2.0:1789760723.175326:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4787 00000100:00100000:2.0:1789760723.175336:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702202775584 time 702253955895 state 0x2 ref 0 00000100:00100000:1.0:1789760723.178511:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760723.178577:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508287488 00000100:00100000:2.0:1789760723.178587:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508287488 00000100:00100000:2.0:1789760723.178637:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702202775584 time 702253955895 state 0x0 ref 1 00000100:00100000:2.0:1789760723.178648:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702254216311 time 702253955895 state 0x1 ref 1 00000100:00100000:2.0:1789760723.178669:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702254216311 time 702283381247 state 0x0 ref 1 00000100:00100000:2.0:1789760723.178678:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508287488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.178714:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19480576, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:2.0:1789760723.178721:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5491b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.178733:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.178737:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.178741:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5491b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.178750:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5491b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760723.199405:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19480576 -> 19484672 00010000:00010000:2.0:1789760723.199424:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5491b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.199436:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5491b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.199451:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5491b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.199457:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.199463:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5491b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.199472:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5491b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.199500:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5491b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.199508:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5491b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19480576->19484671] (req 19480576->19484671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.199609:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508287488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20926us (21102us total) trans 4294972076 rc 0/0 00000100:00100000:2.0:1789760723.199619:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4788 00000100:00100000:2.0:1789760723.199629:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702254216311 time 702283381247 state 0x2 ref 0 00000100:00100000:1.0:1789760723.203696:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760723.204013:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508287616 00000100:00100000:2.0:1789760723.204024:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508287616 00000100:00100000:2.0:1789760723.204077:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702254216311 time 702283381247 state 0x0 ref 1 00000100:00100000:2.0:1789760723.204092:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702283641663 time 702283381247 state 0x1 ref 1 00000100:00100000:2.0:1789760723.204113:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702283641663 time 702308823760 state 0x0 ref 1 00000100:00100000:2.0:1789760723.204136:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508287616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.204184:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19484672, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:2.0:1789760723.204192:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54922 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.204208:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.204222:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.204228:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54922 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.204243:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54922 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760723.223895:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19484672 -> 19488768 00010000:00010000:2.0:1789760723.223912:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54922 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.223924:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54922 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.223934:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54922 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.223942:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.223947:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54922 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.223951:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972077 is committed 00010000:00010000:2.0:1789760723.223956:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54922 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.223978:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54922 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.223986:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54922 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19484672->19488767] (req 19484672->19488767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.224074:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508287616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19947us (20383us total) trans 4294972077 rc 0/0 00000100:00100000:2.0:1789760723.224085:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4789 00000100:00100000:2.0:1789760723.224095:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702283641663 time 702308823760 state 0x2 ref 0 00000100:00100000:0.0:1789760723.228024:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760723.228111:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508287744 00000100:00100000:2.0:1789760723.228122:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508287744 00000100:00100000:2.0:1789760723.228186:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702283641663 time 702308823760 state 0x0 ref 1 00000100:00100000:2.0:1789760723.228200:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702309084176 time 702308823760 state 0x1 ref 1 00000100:00100000:2.0:1789760723.228236:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702309084176 time 702332945770 state 0x0 ref 1 00000100:00100000:2.0:1789760723.228249:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508287744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.228331:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19488768, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:2.0:1789760723.228341:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54929 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.228357:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.228361:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.228367:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54929 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.228381:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54929 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760723.252426:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19488768 -> 19492864 00010000:00010000:2.0:1789760723.252440:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54929 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.252453:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54929 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.252462:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54929 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.252470:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.252475:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54929 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.252552:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54929 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.252587:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54929 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.252595:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54929 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19488768->19492863] (req 19488768->19492863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.252688:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508287744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24437us (24670us total) trans 4294972078 rc 0/0 00000100:00100000:2.0:1789760723.252699:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4790 00000100:00100000:2.0:1789760723.252708:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702309084176 time 702332945770 state 0x2 ref 0 00000100:00100000:1.0:1789760723.257428:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760723.257608:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508287872 00000100:00100000:2.0:1789760723.257619:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508287872 00000100:00100000:2.0:1789760723.257671:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702309084176 time 702332945770 state 0x0 ref 1 00000100:00100000:2.0:1789760723.257686:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702333206186 time 702332945770 state 0x1 ref 1 00000100:00100000:2.0:1789760723.257708:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702333206186 time 702362418695 state 0x0 ref 1 00000100:00100000:2.0:1789760723.257720:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508287872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.257768:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19492864, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:2.0:1789760723.257778:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54930 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.257794:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.257799:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.257806:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54930 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.257820:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54930 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760723.285412:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19492864 -> 19496960 00010000:00010000:2.0:1789760723.285427:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54930 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.285440:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54930 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.285450:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54930 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.285457:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.285463:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54930 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.285471:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54930 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.285492:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54930 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.285500:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54930 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19492864->19496959] (req 19492864->19496959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.285601:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508287872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27878us (28179us total) trans 4294972079 rc 0/0 00000100:00100000:2.0:1789760723.285611:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4791 00000100:00100000:2.0:1789760723.285621:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702333206186 time 702362418695 state 0x2 ref 0 00000100:00100000:1.0:1789760723.291345:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760723.291435:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508288000 00000100:00100000:2.0:1789760723.291454:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508288000 00000100:00100000:2.0:1789760723.291507:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702333206186 time 702362418695 state 0x0 ref 1 00000100:00100000:2.0:1789760723.291523:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702362679111 time 702362418695 state 0x1 ref 1 00000100:00100000:2.0:1789760723.291553:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702362679111 time 702396262978 state 0x0 ref 1 00000100:00100000:2.0:1789760723.291573:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508288000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.291618:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19496960, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:2.0:1789760723.291626:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54937 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.291642:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.291646:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.291652:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54937 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.291667:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54937 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760723.351072:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19496960 -> 19501056 00010000:00010000:2.0:1789760723.351090:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54937 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.351103:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54937 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.351112:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54937 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.351120:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.351125:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54937 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.351135:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54937 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.351157:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54937 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.351165:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54937 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19496960->19501055] (req 19496960->19501055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.351273:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508288000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59696us (59933us total) trans 4294972080 rc 0/0 00000100:00100000:2.0:1789760723.351346:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4792 00000100:00100000:2.0:1789760723.351356:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702362679111 time 702396262978 state 0x2 ref 0 00000100:00100000:0.0:1789760723.357400:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760723.357522:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508288128 00000100:00100000:2.0:1789760723.357532:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508288128 00000100:00100000:2.0:1789760723.357645:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702362679111 time 702396262978 state 0x0 ref 1 00000100:00100000:2.0:1789760723.357663:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702396523394 time 702396262978 state 0x1 ref 1 00000100:00100000:2.0:1789760723.357698:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702396523394 time 702462408369 state 0x0 ref 1 00000100:00100000:2.0:1789760723.357710:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508288128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.357758:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19501056, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:2.0:1789760723.357768:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5493e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.357784:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.357788:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.357794:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5493e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.357807:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5493e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760723.416657:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19501056 -> 19505152 00010000:00010000:2.0:1789760723.416672:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5493e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.416686:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5493e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.416696:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5493e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.416704:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.416708:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5493e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.416717:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5493e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.416735:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5493e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.416743:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5493e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19501056->19505151] (req 19501056->19505151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.416832:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508288128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59120us (59439us total) trans 4294972081 rc 0/0 00000100:00100000:2.0:1789760723.416842:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4793 00000100:00100000:2.0:1789760723.416852:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702396523394 time 702462408369 state 0x2 ref 0 00000100:00100000:1.0:1789760723.424375:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760723.424513:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508288256 00000100:00100000:2.0:1789760723.424523:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508288256 00000100:00100000:2.0:1789760723.424590:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702396523394 time 702462408369 state 0x0 ref 1 00000100:00100000:2.0:1789760723.424605:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702462668785 time 702462408369 state 0x1 ref 1 00000100:00100000:2.0:1789760723.424633:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702462668785 time 702529344132 state 0x0 ref 1 00000100:00100000:2.0:1789760723.424646:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508288256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.424694:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19505152, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:2.0:1789760723.424703:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54945 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.424718:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.424722:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.424729:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54945 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.424743:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54945 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760723.477092:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19505152 -> 19509248 00010000:00010000:2.0:1789760723.477113:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54945 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.477796:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54945 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.477813:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54945 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.477821:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.477825:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54945 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.477834:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54945 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.477854:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54945 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.477862:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54945 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19505152->19509247] (req 19505152->19509247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.477954:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508288256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53306us (53585us total) trans 4294972082 rc 0/0 00000100:00100000:2.0:1789760723.477964:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4794 00000100:00100000:2.0:1789760723.477974:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702462668785 time 702529344132 state 0x2 ref 0 00000001:00080000:0.0:1789760723.480828:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972082 is committed 00000100:00100000:2.0:1789760723.482001:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508288384 00000100:00100000:2.0:1789760723.482013:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508288384 00000100:00100000:2.0:1789760723.482073:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702462668785 time 702529344132 state 0x0 ref 1 00000100:00100000:2.0:1789760723.482090:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702529604548 time 702529344132 state 0x1 ref 1 00000100:00100000:2.0:1789760723.482114:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702529604548 time 702586824454 state 0x0 ref 1 00000100:00100000:2.0:1789760723.482138:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508288384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.482195:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19509248, mtime: 0x6aad94d3, atime: 0x0, ctime: 0x6aad94d3, blocks: 0x9807 00010000:00010000:2.0:1789760723.482215:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5494c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.482232:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.482236:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.482242:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5494c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.482256:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5494c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19509248->19513343] (req 19509248->19513343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.511449:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972083 is committed 00000100:00100000:0.0:1789760723.515732:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760723.515833:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508288512 00000100:00100000:2.0:1789760723.515843:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508288512 00000100:00100000:2.0:1789760723.515893:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702529604548 time 702586824454 state 0x0 ref 1 00000100:00100000:2.0:1789760723.515907:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702587084870 time 702586824454 state 0x1 ref 1 00000100:00100000:2.0:1789760723.515971:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702587084870 time 702620680082 state 0x0 ref 1 00000100:00100000:2.0:1789760723.515984:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508288512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.516036:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19513344, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:2.0:1789760723.516046:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54953 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.516063:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.516068:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.516075:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54953 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.516091:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54953 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19513344->19517439] (req 19513344->19517439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.547857:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972084 is committed 00000100:00100000:1.0:1789760723.551918:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760723.552032:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508288640 00000100:00100000:2.0:1789760723.552044:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508288640 00000100:00100000:2.0:1789760723.552107:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702587084870 time 702620680082 state 0x0 ref 1 00000100:00100000:2.0:1789760723.552197:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702620940498 time 702620680082 state 0x1 ref 1 00000100:00100000:2.0:1789760723.552280:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702620940498 time 702656989798 state 0x0 ref 1 00000100:00100000:2.0:1789760723.552325:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508288640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.552373:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19517440, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:2.0:1789760723.552382:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5495a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.552398:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.552403:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.552408:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5495a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.552423:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5495a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760723.578414:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972085 is committed 00002000:00010000:2.0:1789760723.578483:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19517440 -> 19521536 00010000:00010000:2.0:1789760723.578497:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5495a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.578509:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5495a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.578517:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5495a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.578524:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.578529:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5495a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.578537:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5495a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.578568:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5495a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.578576:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5495a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19517440->19521535] (req 19517440->19521535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.578661:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508288640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26333us (26752us total) trans 4294972085 rc 0/0 00000100:00100000:2.0:1789760723.578670:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4797 00000100:00100000:2.0:1789760723.578680:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702620940498 time 702656989798 state 0x2 ref 0 00000100:00100000:0.0:1789760723.582473:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760723.604264:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972086 is committed 00002000:00010000:1.0:1789760723.604405:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19521536 -> 19525632 00010000:00010000:1.0:1789760723.604420:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54961 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.604435:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54961 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.604443:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54961 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.604451:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760723.604456:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54961 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.604466:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54961 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.604490:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54961 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.604499:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54961 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19521536->19525631] (req 19521536->19525631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760723.604599:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508288768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 21923us (22130us total) trans 4294972086 rc 0/0 00000100:00100000:1.0:1789760723.604609:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4798 00000100:00100000:1.0:1789760723.604620:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702657250214 time 702687374626 state 0x2 ref 0 00000100:00100000:2.0:1789760723.608174:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760723.638766:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972087 is committed 00000100:00100000:2.0:1789760723.642357:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760723.642464:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508289024 00000100:00100000:0.0:1789760723.642473:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508289024 00000100:00100000:0.0:1789760723.642526:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702687635042 time 702713117620 state 0x0 ref 1 00000100:00100000:0.0:1789760723.642541:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702713378036 time 702713117620 state 0x1 ref 1 00000100:00100000:0.0:1789760723.642567:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702713378036 time 702747271670 state 0x0 ref 1 00000100:00100000:0.0:1789760723.642579:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508289024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760723.642628:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19529728, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:0.0:1789760723.642637:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5496f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.642651:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760723.642655:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.642662:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5496f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.642675:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5496f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.666994:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972088 is committed 00002000:00010000:1.0:1789760723.667429:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19529728 -> 19533824 00010000:00010000:1.0:1789760723.667443:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5496f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.667457:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5496f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.667464:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5496f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.667472:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760723.667476:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5496f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.667490:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5496f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.667509:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5496f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760723.667515:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5496f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19529728->19533823] (req 19529728->19533823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760723.668699:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508289024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26113us (26343us total) trans 4294972088 rc 0/0 00000100:00100000:0.0:1789760723.668711:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4800 00000100:00100000:0.0:1789760723.668720:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702713378036 time 702747271670 state 0x2 ref 0 00000100:00100000:0.0:1789760723.671818:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760723.671895:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508289152 00000100:00100000:1.0:1789760723.671904:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508289152 00000100:00100000:1.0:1789760723.671955:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702713378036 time 702747271670 state 0x0 ref 1 00000100:00100000:1.0:1789760723.671970:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702747532086 time 702747271670 state 0x1 ref 1 00000100:00100000:0.0:1789760723.672083:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702747532086 time 702776791688 state 0x0 ref 1 00000100:00100000:0.0:1789760723.672097:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508289152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760723.672223:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19533824, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:0.0:1789760723.672231:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54976 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.672251:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760723.672255:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.672261:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54976 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.672274:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54976 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.722338:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760723.729508:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19533824 -> 19537920 00010000:00010000:2.0:1789760723.729526:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54976 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.729540:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54976 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.729551:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54976 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.729572:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.729577:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54976 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.729586:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54976 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.729614:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54976 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.729621:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54976 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19533824->19537919] (req 19533824->19537919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.729724:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508289152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57624us (57911us total) trans 4294972089 rc 0/0 00000100:00100000:2.0:1789760723.729735:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4801 00000100:00100000:2.0:1789760723.729745:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702747532086 time 702776791688 state 0x2 ref 0 00000001:00080000:1.0:1789760723.730262:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972089 is committed 00000100:00100000:1.0:1789760723.734012:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760723.781089:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972090 is committed 00000100:00100000:0.0:1789760723.784938:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760723.785079:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508289408 00000100:00100000:2.0:1789760723.785089:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508289408 00000100:00100000:2.0:1789760723.785215:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702777052104 time 702838929963 state 0x0 ref 1 00000100:00100000:2.0:1789760723.785231:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702839190379 time 702838929963 state 0x1 ref 1 00000100:00100000:2.0:1789760723.785263:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702839190379 time 702889973815 state 0x0 ref 1 00000100:00100000:2.0:1789760723.785275:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508289408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.785366:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19542016, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:2.0:1789760723.785375:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54984 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.785393:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:2.0:1789760723.785398:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.785405:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54984 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.785420:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54984 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19542016->19546111] (req 19542016->19546111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760723.812525:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972091 is committed 00000100:00100000:2.0:1789760723.816872:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760723.816975:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508289536 00000100:00100000:0.0:1789760723.816984:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508289536 00000100:00100000:0.0:1789760723.817039:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702839190379 time 702889973815 state 0x0 ref 1 00000100:00100000:0.0:1789760723.817054:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702890234231 time 702889973815 state 0x1 ref 1 00000100:00100000:0.0:1789760723.817131:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702890234231 time 702921841115 state 0x0 ref 1 00000100:00100000:0.0:1789760723.817144:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508289536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760723.817190:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19546112, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:0.0:1789760723.817199:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5498b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.817226:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=702s v=(0 0) 00010000:00010000:0.0:1789760723.817230:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760723.817237:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5498b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760723.817251:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5498b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19546112->19550207] (req 19546112->19550207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.838377:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04985468a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534061568:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760723.838450:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760723.838530:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534061568 00000100:00100000:2.0:1789760723.838538:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534061568 00000100:00100000:2.0:1789760723.838593:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:2.0:1789760723.838602:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534061568:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760723.838640:0:7358:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789760723.838687:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534061568:12345-0@lo:41:osp-pre-1-0.0 Request processed in 85us (237us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760723.838695:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00100000:2.0:1789760723.838738:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04985468a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534061568:0@lo:41:osp-pre-1-0.0 00000001:00080000:1.0:1789760723.855473:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972092 is committed 00000100:00100000:1.0:1789760723.859985:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:1.0:1789760723.898347:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972093 is committed 00002000:00010000:2.0:1789760723.899851:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19550208 -> 19554304 00010000:00010000:2.0:1789760723.899870:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54992 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.899882:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54992 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.899891:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54992 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.899899:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760723.899904:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54992 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.899914:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54992 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.899937:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54992 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.899946:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54992 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19550208->19554303] (req 19550208->19554303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760723.900073:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508289664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39828us (40092us total) trans 4294972093 rc 0/0 00000100:00100000:2.0:1789760723.900085:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4805 00000100:00100000:2.0:1789760723.900096:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702922101531 time 702964940023 state 0x2 ref 0 00000100:00100000:0.0:1789760723.906237:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760723.906711:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702922101531 time 702964940023 state 0x0 ref 1 00000100:00100000:2.0:1789760723.906727:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702965200439 time 702964940023 state 0x1 ref 1 00000100:00100000:2.0:1789760723.906797:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 702965200439 time 703011506242 state 0x0 ref 1 00000100:00100000:2.0:1789760723.906810:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508289792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.906859:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19554304, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:2.0:1789760723.906868:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54999 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.906883:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760723.906887:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.906893:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54999 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.906906:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54999 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19554304->19558399] (req 19554304->19558399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760723.955086:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972094 is committed 00000100:00100000:1.0:1789760723.957652:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:2.0:1789760723.985960:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972095 is committed 00000100:00100000:1.0:1789760723.990570:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760723.990808:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508290048 00000100:00100000:2.0:1789760723.990819:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508290048 00000100:00100000:2.0:1789760723.990872:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703011766658 time 703062592833 state 0x0 ref 1 00000100:00100000:2.0:1789760723.990888:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703062853249 time 703062592833 state 0x1 ref 1 00000100:00100000:2.0:1789760723.990949:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703062853249 time 703095659585 state 0x0 ref 1 00000100:00100000:2.0:1789760723.990958:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508290048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760723.990998:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19562496, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:2.0:1789760723.991007:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.991027:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760723.991032:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760723.991038:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760723.991050:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.027449:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972096 is committed 00002000:00010000:2.0:1789760724.028068:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19562496 -> 19566592 00010000:00010000:2.0:1789760724.028083:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.028097:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.028107:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.028125:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760724.028131:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.028140:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.028163:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.028172:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19562496->19566591] (req 19562496->19566591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.028265:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508290048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37302us (37710us total) trans 4294972096 rc 0/0 00000100:00100000:2.0:1789760724.028275:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4808 00000100:00100000:2.0:1789760724.028316:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703062853249 time 703095659585 state 0x2 ref 0 00000100:00100000:0.0:1789760724.041113:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760724.041466:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703095920001 time 703146174716 state 0x0 ref 1 00000100:00100000:2.0:1789760724.041482:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508290176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760724.041530:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19566592, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:2.0:1789760724.041539:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.041729:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.041735:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.041742:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.041757:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.100072:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972097 is committed 00002000:00010000:0.0:1789760724.100190:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19566592 -> 19570688 00010000:00010000:0.0:1789760724.100216:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.100229:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.100238:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.100245:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760724.100250:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.100259:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.100281:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.100317:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19566592->19570687] (req 19566592->19570687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.100403:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508290176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58919us (59295us total) trans 4294972097 rc 0/0 00000100:00100000:0.0:1789760724.100413:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4809 00000100:00100000:0.0:1789760724.100424:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703095920001 time 703146174716 state 0x2 ref 0 00000100:00100000:1.0:1789760724.112243:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760724.169031:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972098 is committed 00000100:00100000:1.0:1789760724.174021:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760724.177115:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508290432 00000100:00100000:2.0:1789760724.177131:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508290432 00000100:00100000:2.0:1789760724.177193:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703146435132 time 703218559177 state 0x0 ref 1 00000100:00100000:2.0:1789760724.177220:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703218819593 time 703218559177 state 0x1 ref 1 00000100:00100000:2.0:1789760724.177325:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703218819593 time 703282034246 state 0x0 ref 1 00000100:00100000:2.0:1789760724.177340:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508290432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760724.177387:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19574784, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:2.0:1789760724.177395:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.177411:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.177415:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.177421:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.177436:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760724.217495:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19574784 -> 19578880 00010000:00010000:2.0:1789760724.217511:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.217524:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.217540:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.217547:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760724.217618:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.217629:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.217651:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.217659:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe549bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19574784->19578879] (req 19574784->19578879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.217829:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508290432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40486us (43813us total) trans 4294972099 rc 0/0 00000100:00100000:2.0:1789760724.217840:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4811 00000100:00100000:2.0:1789760724.217849:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703218819593 time 703282034246 state 0x2 ref 0 00000100:00100000:1.0:1789760724.222350:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760724.222433:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508290560 00000100:00100000:0.0:1789760724.222451:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508290560 00000100:00100000:0.0:1789760724.222505:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703218819593 time 703282034246 state 0x0 ref 1 00000100:00100000:0.0:1789760724.222520:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703282294662 time 703282034246 state 0x1 ref 1 00000100:00100000:0.0:1789760724.222546:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703282294662 time 703327256874 state 0x0 ref 1 00000100:00100000:0.0:1789760724.222565:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508290560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760724.222610:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19578880, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:0.0:1789760724.222619:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe549c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.222635:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760724.222640:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.222646:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe549c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.222666:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe549c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760724.249380:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19578880 -> 19582976 00010000:00010000:0.0:1789760724.249400:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe549c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.249412:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe549c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.249421:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe549c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.249428:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760724.249433:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe549c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.249442:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe549c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.249461:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe549c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.249468:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe549c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19578880->19582975] (req 19578880->19582975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.249793:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508290560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27225us (27447us total) trans 4294972100 rc 0/0 00000100:00100000:0.0:1789760724.249804:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4812 00000100:00100000:0.0:1789760724.249814:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703282294662 time 703327256874 state 0x2 ref 0 00000001:00080000:0.0:1789760724.250103:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972100 is committed 00000100:00100000:0.0:1789760724.254059:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508290688 00000100:00100000:0.0:1789760724.254068:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508290688 00000100:00100000:0.0:1789760724.254165:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703282294662 time 703327256874 state 0x0 ref 1 00000100:00100000:0.0:1789760724.254177:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703327517290 time 703327256874 state 0x1 ref 1 00000100:00100000:0.0:1789760724.254248:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703327517290 time 703358958820 state 0x0 ref 1 00000100:00100000:0.0:1789760724.254258:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508290688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760724.254324:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19582976, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:0.0:1789760724.254333:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.254348:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760724.254353:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.254364:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.254377:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760724.280878:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972101 is committed 00002000:00010000:0.0:1789760724.280979:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19582976 -> 19587072 00010000:00010000:0.0:1789760724.280995:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.281009:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.281018:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.281025:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760724.281030:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.281040:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.281061:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.281070:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19582976->19587071] (req 19582976->19587071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.281168:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508290688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26905us (27428us total) trans 4294972101 rc 0/0 00000100:00100000:0.0:1789760724.281182:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4813 00000100:00100000:0.0:1789760724.281192:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703327517290 time 703358958820 state 0x2 ref 0 00000100:00100000:2.0:1789760724.284998:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508290816 00000100:00100000:2.0:1789760724.285008:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508290816 00000100:00100000:2.0:1789760724.285063:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703327517290 time 703358958820 state 0x0 ref 1 00000100:00100000:2.0:1789760724.285082:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703359219236 time 703358958820 state 0x1 ref 1 00000100:00100000:2.0:1789760724.285115:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703359219236 time 703389825406 state 0x0 ref 1 00000100:00100000:2.0:1789760724.285128:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508290816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760724.285177:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19587072, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:2.0:1789760724.285186:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe549d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.285210:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.285215:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.285222:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe549d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.285237:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe549d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19587072->19591167] (req 19587072->19591167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760724.317660:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972102 is committed 00000100:00100000:2.0:1789760724.322119:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760724.322241:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508290944 00000100:00100000:0.0:1789760724.322252:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508290944 00000100:00100000:0.0:1789760724.322343:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703359219236 time 703389825406 state 0x0 ref 1 00000100:00100000:0.0:1789760724.322358:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703390085822 time 703389825406 state 0x1 ref 1 00000100:00100000:0.0:1789760724.322379:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703390085822 time 703427089734 state 0x0 ref 1 00000100:00100000:0.0:1789760724.322390:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508290944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760724.322436:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19591168, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:0.0:1789760724.322445:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.322460:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760724.322465:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.322471:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.322484:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.347470:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972103 is committed 00002000:00010000:0.0:1789760724.347770:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19591168 -> 19595264 00010000:00010000:0.0:1789760724.348604:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.348619:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.348628:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.348636:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760724.348640:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.348649:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.348669:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.348677:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19591168->19595263] (req 19591168->19595263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.348876:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508290944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26483us (26762us total) trans 4294972103 rc 0/0 00000100:00100000:0.0:1789760724.348887:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4815 00000100:00100000:0.0:1789760724.348896:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703390085822 time 703427089734 state 0x2 ref 0 00000100:00100000:2.0:1789760724.355880:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760724.356033:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508291072 00000100:00100000:0.0:1789760724.356044:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508291072 00000100:00100000:0.0:1789760724.356100:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703390085822 time 703427089734 state 0x0 ref 1 00000100:00100000:0.0:1789760724.356116:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703427350150 time 703427089734 state 0x1 ref 1 00000100:00100000:0.0:1789760724.356150:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703427350150 time 703460860265 state 0x0 ref 1 00000100:00100000:0.0:1789760724.356163:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d46450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508291072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760724.356221:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19595264, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:0.0:1789760724.356231:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.356246:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760724.356251:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.356257:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.356271:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760724.397402:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19595264 -> 19599360 00010000:00010000:0.0:1789760724.397419:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.397501:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.397517:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.397525:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760724.397530:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.397538:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.397619:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.397628:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19595264->19599359] (req 19595264->19599359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.397766:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d46450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508291072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41601us (41891us total) trans 4294972104 rc 0/0 00000100:00100000:0.0:1789760724.397785:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4816 00000100:00100000:0.0:1789760724.397795:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703427350150 time 703460860265 state 0x2 ref 0 00000100:00100000:2.0:1789760724.401599:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508291200 00000100:00100000:2.0:1789760724.401609:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508291200 00000100:00100000:2.0:1789760724.401662:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703427350150 time 703460860265 state 0x0 ref 1 00000100:00100000:2.0:1789760724.401677:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703461120681 time 703460860265 state 0x1 ref 1 00000100:00100000:2.0:1789760724.401713:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703461120681 time 703506423299 state 0x0 ref 1 00000100:00100000:2.0:1789760724.401733:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508291200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760724.401788:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19599360, mtime: 0x6aad94d4, atime: 0x0, ctime: 0x6aad94d4, blocks: 0x9807 00010000:00010000:2.0:1789760724.401797:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe549e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.401810:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.401815:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.401820:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe549e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.401834:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe549e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.480782:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972105 is committed 00002000:00010000:0.0:1789760724.480984:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19599360 -> 19603456 00010000:00010000:0.0:1789760724.481002:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe549e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.481022:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe549e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.481031:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe549e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.481039:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760724.481045:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe549e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.481055:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe549e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.481078:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe549e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.481085:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe549e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19599360->19603455] (req 19599360->19603455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.481190:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508291200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 79454us (79755us total) trans 4294972105 rc 0/0 00000100:00100000:0.0:1789760724.481215:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4817 00000100:00100000:0.0:1789760724.481226:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703461120681 time 703506423299 state 0x2 ref 0 00000100:00100000:0.0:1789760724.488861:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760724.489143:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508291328 00000100:00100000:2.0:1789760724.489154:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508291328 00000100:00100000:2.0:1789760724.489224:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703461120681 time 703506423299 state 0x0 ref 1 00000100:00100000:2.0:1789760724.489241:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703506683715 time 703506423299 state 0x1 ref 1 00000100:00100000:2.0:1789760724.489274:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703506683715 time 703593983753 state 0x0 ref 1 00000100:00100000:2.0:1789760724.489507:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508291328:12345-192.168.203.23@tcp:4:dd.0 00000001:00080000:1.0:1789760724.558757:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972106 is committed 00000100:00100000:2.0:1789760724.563209:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760724.563425:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508291456 00000100:00100000:0.0:1789760724.563435:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508291456 00000100:00100000:0.0:1789760724.563489:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703506683715 time 703593983753 state 0x0 ref 1 00000100:00100000:0.0:1789760724.563504:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703594244169 time 703593983753 state 0x1 ref 1 00000100:00100000:0.0:1789760724.563524:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703594244169 time 703668235263 state 0x0 ref 1 00000100:00100000:0.0:1789760724.563536:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508291456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760724.563593:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19607552, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:0.0:1789760724.563602:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.563617:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760724.563623:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.563629:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.563643:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760724.595663:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972107 is committed 00002000:00010000:0.0:1789760724.595744:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19607552 -> 19611648 00010000:00010000:0.0:1789760724.595759:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.595772:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.595787:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.595794:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760724.595799:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.595807:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.595827:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.595836:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19607552->19611647] (req 19607552->19611647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760724.595931:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508291456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32391us (32737us total) trans 4294972107 rc 0/0 00000100:00100000:0.0:1789760724.595942:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4819 00000100:00100000:0.0:1789760724.595951:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703594244169 time 703668235263 state 0x2 ref 0 00000100:00100000:1.0:1789760724.601637:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760724.601833:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508291584 00000100:00100000:0.0:1789760724.601844:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508291584 00000100:00100000:0.0:1789760724.601902:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703594244169 time 703668235263 state 0x0 ref 1 00000100:00100000:0.0:1789760724.601917:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703668495679 time 703668235263 state 0x1 ref 1 00000100:00100000:0.0:1789760724.601939:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703668495679 time 703706650115 state 0x0 ref 1 00000100:00100000:0.0:1789760724.601951:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508291584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760724.602000:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19611648, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:0.0:1789760724.602009:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.602024:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:0.0:1789760724.602029:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760724.602035:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.602049:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760724.711881:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19611648 -> 19615744 00010000:00010000:2.0:1789760724.711896:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.711909:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.711918:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.711925:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760724.711929:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.711938:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.711956:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.711964:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe549fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19611648->19615743] (req 19611648->19615743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760724.712057:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508291584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 110104us (110425us total) trans 4294972108 rc 0/0 00000100:00100000:2.0:1789760724.712067:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4820 00000100:00100000:2.0:1789760724.712077:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703668495679 time 703706650115 state 0x2 ref 0 00000100:00100000:1.0:1789760724.717066:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00010000:00010000:3.0:1789760724.717663:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760724.734772:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760724.776972:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19615744 -> 19619840 00010000:00010000:3.0:1789760724.776993:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.777007:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.777017:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.777024:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760724.777029:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.777037:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.777057:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.777066:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19615744->19619839] (req 19615744->19619839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.777196:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508291712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59711us (60133us total) trans 4294972109 rc 0/0 00000100:00100000:3.0:1789760724.777222:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4821 00000100:00100000:3.0:1789760724.777232:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703706910531 time 703822177347 state 0x2 ref 0 00000001:00080000:3.0:1789760724.777276:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972109 is committed 00000100:00100000:3.0:1789760724.781652:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760724.785430:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508291840 00000100:00100000:0.0:1789760724.785453:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508291840 00000100:00100000:1.0:1789760724.785678:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703706910531 time 703822177347 state 0x0 ref 1 00000100:00100000:1.0:1789760724.785698:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703822437763 time 703822177347 state 0x1 ref 1 00000100:00100000:1.0:1789760724.786034:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703822437763 time 703890742352 state 0x0 ref 1 00000100:00100000:1.0:1789760724.786052:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508291840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760724.786116:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19619840, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:1.0:1789760724.786128:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54a09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.786157:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:1.0:1789760724.786163:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760724.786171:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54a09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.786190:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54a09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760724.867314:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19619840 -> 19623936 00010000:00010000:1.0:1789760724.867334:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54a09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.867348:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54a09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.867357:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54a09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.867377:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760724.867383:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54a09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.867392:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54a09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.867413:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54a09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760724.867420:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54a09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19619840->19623935] (req 19619840->19623935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760724.867530:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508291840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 81478us (85977us total) trans 4294972110 rc 0/0 00000100:00100000:1.0:1789760724.867542:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4822 00000100:00100000:1.0:1789760724.867853:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703822437763 time 703890742352 state 0x2 ref 0 00000001:00080000:3.0:1789760724.868311:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972110 is committed 00000100:00100000:1.0:1789760724.872971:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760724.873059:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508291968 00000100:00100000:2.0:1789760724.873070:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508291968 00000100:00100000:2.0:1789760724.873128:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703822437763 time 703890742352 state 0x0 ref 1 00000100:00100000:2.0:1789760724.873144:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703891002768 time 703890742352 state 0x1 ref 1 00000100:00100000:2.0:1789760724.873178:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703891002768 time 703977887716 state 0x0 ref 1 00000100:00100000:2.0:1789760724.873193:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508291968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760724.873264:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19623936, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:2.0:1789760724.873274:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54a10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.873308:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=703s v=(0 0) 00010000:00010000:2.0:1789760724.873313:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760724.873319:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54a10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760724.873334:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54a10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760724.945089:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19623936 -> 19628032 00010000:00010000:3.0:1789760724.945107:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54a10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.945121:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54a10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.945131:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54a10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.945138:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760724.945143:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54a10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.945152:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54a10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.945233:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54a10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.945244:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe54a10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19623936->19628031] (req 19623936->19628031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.945430:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972111 is committed 00000100:00100000:3.0:1789760724.945689:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508291968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 72494us (72722us total) trans 4294972111 rc 0/0 00000100:00100000:3.0:1789760724.945700:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4823 00000100:00100000:3.0:1789760724.945709:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703891002768 time 703977887716 state 0x2 ref 0 00000100:00100000:2.0:1789760724.948095:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760724.948172:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508292096 00000100:00100000:3.0:1789760724.948235:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508292096 00000100:00100000:3.0:1789760724.948305:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703891002768 time 703977887716 state 0x0 ref 1 00000100:00100000:3.0:1789760724.948320:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703978148132 time 703977887716 state 0x1 ref 1 00000100:00100000:3.0:1789760724.948470:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703978148132 time 704053179804 state 0x0 ref 1 00000100:00100000:3.0:1789760724.948484:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508292096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760724.948529:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19628032, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:3.0:1789760724.948537:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.948552:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760724.948557:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760724.948562:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.948575:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760724.999193:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19628032 -> 19632128 00010000:00010000:3.0:1789760724.999224:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760724.999232:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972112 is committed 00010000:00010000:3.0:1789760724.999267:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.999277:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.999332:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760724.999338:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.999348:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.999368:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760724.999376:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19628032->19632127] (req 19628032->19632127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760724.999487:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508292096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51001us (51396us total) trans 4294972112 rc 0/0 00000100:00100000:3.0:1789760724.999501:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4824 00000100:00100000:3.0:1789760724.999511:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703978148132 time 704053179804 state 0x2 ref 0 00000100:00100000:0.0:1789760725.003632:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760725.003733:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508292224 00000100:00100000:1.0:1789760725.003743:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508292224 00000100:00100000:1.0:1789760725.003814:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 703978148132 time 704053179804 state 0x0 ref 1 00000100:00100000:1.0:1789760725.003830:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704053440220 time 704053179804 state 0x1 ref 1 00000100:00100000:1.0:1789760725.003862:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704053440220 time 704108572697 state 0x0 ref 1 00000100:00100000:1.0:1789760725.003880:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508292224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760725.003929:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19632128, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:1.0:1789760725.003938:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54a1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.003954:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789760725.003959:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760725.003965:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54a1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.003979:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54a1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760725.004488:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760725.036800:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972113 is committed 00002000:00010000:3.0:1789760725.036920:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19632128 -> 19636224 00010000:00010000:3.0:1789760725.036936:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54a1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.036950:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54a1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.036959:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54a1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.036967:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760725.036972:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54a1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.036982:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54a1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.037002:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54a1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.037010:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54a1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19632128->19636223] (req 19632128->19636223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.037113:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508292224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33235us (33488us total) trans 4294972113 rc 0/0 00000100:00100000:3.0:1789760725.037125:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4825 00000100:00100000:3.0:1789760725.037135:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704053440220 time 704108572697 state 0x2 ref 0 00000100:00100000:1.0:1789760725.042953:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760725.044275:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508292352 00000100:00100000:2.0:1789760725.044325:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508292352 00000100:00100000:2.0:1789760725.044389:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704053440220 time 704108572697 state 0x0 ref 1 00000100:00100000:2.0:1789760725.044408:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704108833113 time 704108572697 state 0x1 ref 1 00000100:00100000:2.0:1789760725.044440:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704108833113 time 704149150569 state 0x0 ref 1 00000100:00100000:2.0:1789760725.044454:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508292352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760725.044512:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19636224, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:2.0:1789760725.044522:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.044537:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760725.044542:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.044561:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.044575:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.061435:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760725.061488:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760725.061682:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508292608 00000100:00100000:1.0:1789760725.061693:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508292608 00000100:00100000:1.0:1789760725.061770:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 859 00000100:00100000:1.0:1789760725.061783:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508292608:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760725.061863:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508292608:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 85us (445us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760725.061871:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 859 00000100:00100000:1.0:1789760725.061975:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508292480 00000100:00100000:1.0:1789760725.061980:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508292480 00000100:00100000:1.0:1789760725.062042:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 465 00000100:00100000:1.0:1789760725.062049:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04989031050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2409:x1876699508292480:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760725.062075:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04989031050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2409:x1876699508292480:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 27us (588us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760725.062082:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 465 00000100:00080000:3.0:1789760725.062209:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760725.062412:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760725.062437:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760725.062532:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508292736 00000100:00100000:3.0:1789760725.062537:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508292736 00000100:00100000:2.0:1789760725.062623:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760725.062656:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508292992 00000100:00100000:2.0:1789760725.062661:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508292992 00000100:00100000:0.0:1789760725.062665:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508292864 00000100:00100000:0.0:1789760725.062671:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508292864 00000100:00100000:2.0:1789760725.062709:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 860 00000100:00100000:2.0:1789760725.062718:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508292992:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760725.062761:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508292992:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 44us (323us total) trans 0 rc 0/0 00000100:00080000:1.0:1789760725.063156:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760725.063161:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760725.063211:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 861 00000100:00100000:0.0:1789760725.063222:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508292864:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760725.063254:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508292864:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 35us (633us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760725.063261:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 861 00000100:00100000:2.0:1789760725.064275:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 860 00000100:00100000:1.0:1789760725.066235:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 862 00000100:00100000:1.0:1789760725.066255:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508292736:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760725.066373:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508292736:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 125us (3961us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760725.066380:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 862 00000001:00080000:2.0:1789760725.093096:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972114 is committed 00002000:00010000:2.0:1789760725.093157:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19636224 -> 19640320 00010000:00010000:2.0:1789760725.093171:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.093185:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.093203:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.093210:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760725.093215:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.093224:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.093242:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.093249:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19636224->19640319] (req 19636224->19640319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.093386:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508292352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48930us (50439us total) trans 4294972114 rc 0/0 00000100:00100000:2.0:1789760725.093397:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4826 00000100:00100000:2.0:1789760725.093411:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704108833113 time 704149150569 state 0x2 ref 0 00000100:00100000:0.0:1789760725.097648:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760725.097908:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508293120 00000100:00100000:3.0:1789760725.097925:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508293120 00000100:00100000:3.0:1789760725.098008:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704108833113 time 704149150569 state 0x0 ref 1 00000100:00100000:3.0:1789760725.098028:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704149410985 time 704149150569 state 0x1 ref 1 00000100:00100000:3.0:1789760725.098069:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704149410985 time 704202779522 state 0x0 ref 1 00000100:00100000:3.0:1789760725.098088:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508293120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760725.098169:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19640320, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:3.0:1789760725.098180:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.098214:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760725.098220:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.098228:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.098251:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760725.153316:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19640320 -> 19644416 00010000:00010000:2.0:1789760725.153344:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.153360:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.153370:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.153379:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760725.153385:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.153396:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.153424:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.153432:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54a2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19640320->19644415] (req 19640320->19644415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.153703:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508293120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55612us (56057us total) trans 4294972115 rc 0/0 00000100:00100000:2.0:1789760725.153715:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4827 00000100:00100000:2.0:1789760725.153728:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704149410985 time 704202779522 state 0x2 ref 0 00000001:00080000:2.0:1789760725.153779:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972115 is committed 00000100:00100000:0.0:1789760725.159166:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760725.159327:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508293248 00000100:00100000:3.0:1789760725.159340:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508293248 00000100:00100000:3.0:1789760725.159410:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704149410985 time 704202779522 state 0x0 ref 1 00000100:00100000:3.0:1789760725.159428:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704203039938 time 704202779522 state 0x1 ref 1 00000100:00100000:3.0:1789760725.159464:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704203039938 time 704264174650 state 0x0 ref 1 00000100:00100000:3.0:1789760725.159477:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508293248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760725.159537:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19644416, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:3.0:1789760725.159556:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54a33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.159574:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760725.159578:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.159586:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54a33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.159602:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54a33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760725.199052:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19644416 -> 19648512 00010000:00010000:0.0:1789760725.199084:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54a33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.199099:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54a33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.199109:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54a33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.199117:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760725.199123:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54a33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.199134:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54a33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.199161:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54a33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.199168:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54a33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19644416->19648511] (req 19644416->19648511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.199758:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508293248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40273us (40596us total) trans 4294972116 rc 0/0 00000100:00100000:0.0:1789760725.199772:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4828 00000100:00100000:0.0:1789760725.199784:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704203039938 time 704264174650 state 0x2 ref 0 00000001:00080000:2.0:1789760725.201509:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972116 is committed 00000100:00100000:1.0:1789760725.205900:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760725.206011:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508293376 00000100:00100000:2.0:1789760725.206026:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508293376 00000100:00100000:2.0:1789760725.206102:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704203039938 time 704264174650 state 0x0 ref 1 00000100:00100000:2.0:1789760725.206121:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704264435066 time 704264174650 state 0x1 ref 1 00000100:00100000:2.0:1789760725.206155:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704264435066 time 704310865267 state 0x0 ref 1 00000100:00100000:2.0:1789760725.206169:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508293376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760725.206245:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19648512, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:2.0:1789760725.206256:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.206277:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760725.206282:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.206342:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.206363:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.286787:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972117 is committed 00002000:00010000:3.0:1789760725.286899:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19648512 -> 19652608 00010000:00010000:3.0:1789760725.286920:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.286935:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.286947:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.286955:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760725.286961:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.287123:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.287148:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.287156:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54a3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19648512->19652607] (req 19648512->19652607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.287470:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508293376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 81297us (81578us total) trans 4294972117 rc 0/0 00000100:00100000:3.0:1789760725.287484:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4829 00000100:00100000:3.0:1789760725.287497:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704264435066 time 704310865267 state 0x2 ref 0 00000100:00100000:1.0:1789760725.303851:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760725.304061:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508293504 00000100:00100000:2.0:1789760725.304076:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508293504 00000100:00100000:2.0:1789760725.304241:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704264435066 time 704310865267 state 0x0 ref 1 00000100:00100000:2.0:1789760725.304263:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704311125683 time 704310865267 state 0x1 ref 1 00000100:00100000:2.0:1789760725.304322:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704311125683 time 704409031522 state 0x0 ref 1 00000100:00100000:2.0:1789760725.304337:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508293504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760725.304406:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19652608, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:2.0:1789760725.304416:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.304436:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760725.304441:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.304449:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.304466:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.391498:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972118 is committed 00002000:00010000:2.0:1789760725.391693:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19652608 -> 19656704 00010000:00010000:2.0:1789760725.391715:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.391731:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.391742:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.391750:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760725.391756:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.391766:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.391794:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.391802:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19652608->19656703] (req 19652608->19656703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.391957:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508293504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 87617us (88117us total) trans 4294972118 rc 0/0 00000100:00100000:2.0:1789760725.391969:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4830 00000100:00100000:2.0:1789760725.391981:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704311125683 time 704409031522 state 0x2 ref 0 00000100:00100000:1.0:1789760725.396427:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760725.396548:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508293632 00000100:00100000:0.0:1789760725.396561:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508293632 00000100:00100000:0.0:1789760725.396634:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704311125683 time 704409031522 state 0x0 ref 1 00000100:00100000:0.0:1789760725.396653:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704409291938 time 704409031522 state 0x1 ref 1 00000100:00100000:0.0:1789760725.396688:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704409291938 time 704501398840 state 0x0 ref 1 00000100:00100000:0.0:1789760725.396703:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508293632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760725.396765:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19656704, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:0.0:1789760725.396777:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.396796:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760725.396801:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.396808:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.396825:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.447163:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972119 is committed 00002000:00010000:2.0:1789760725.447450:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19656704 -> 19660800 00010000:00010000:2.0:1789760725.447605:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.447624:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.447635:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.447643:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760725.447650:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.447660:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.447687:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.447696:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19656704->19660799] (req 19656704->19660799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760725.447945:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508293632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51238us (51527us total) trans 4294972119 rc 0/0 00000100:00100000:2.0:1789760725.447962:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4831 00000100:00100000:2.0:1789760725.447974:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704409291938 time 704501398840 state 0x2 ref 0 00000100:00100000:1.0:1789760725.452737:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760725.452851:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508293760 00000100:00100000:3.0:1789760725.452864:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508293760 00000100:00100000:3.0:1789760725.452936:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704409291938 time 704501398840 state 0x0 ref 1 00000100:00100000:3.0:1789760725.452955:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704501659256 time 704501398840 state 0x1 ref 1 00000100:00100000:3.0:1789760725.453019:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704501659256 time 704557728702 state 0x0 ref 1 00000100:00100000:3.0:1789760725.453035:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508293760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760725.453100:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19660800, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:3.0:1789760725.453111:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.453131:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760725.453136:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.453143:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.453158:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760725.508507:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19660800 -> 19664896 00010000:00010000:3.0:1789760725.508608:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.508632:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.508642:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.508650:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760725.508656:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.508667:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.508693:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.508701:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19660800->19664895] (req 19660800->19664895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.508825:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508293760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55786us (56098us total) trans 4294972120 rc 0/0 00000100:00100000:3.0:1789760725.508838:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4832 00000100:00100000:3.0:1789760725.508851:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704501659256 time 704557728702 state 0x2 ref 0 00000001:00080000:2.0:1789760725.509462:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972120 is committed 00000100:00100000:1.0:1789760725.515604:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760725.515726:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508293888 00000100:00100000:2.0:1789760725.515740:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508293888 00000100:00100000:2.0:1789760725.515817:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704501659256 time 704557728702 state 0x0 ref 1 00000100:00100000:2.0:1789760725.515836:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704557989118 time 704557728702 state 0x1 ref 1 00000100:00100000:2.0:1789760725.515870:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704557989118 time 704620580338 state 0x0 ref 1 00000100:00100000:2.0:1789760725.515884:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508293888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760725.515956:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19664896, mtime: 0x6aad94d5, atime: 0x0, ctime: 0x6aad94d5, blocks: 0x9807 00010000:00010000:2.0:1789760725.515967:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.515988:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:2.0:1789760725.515992:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760725.516000:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760725.516022:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760725.576010:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972121 is committed 00002000:00010000:0.0:1789760725.576445:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19664896 -> 19668992 00002000:00010000:0.0:1789760725.576452:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760725 -> 1789760726 00002000:00010000:0.0:1789760725.576456:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760725 -> 1789760726 00010000:00010000:0.0:1789760725.576477:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.576494:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.576506:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.576516:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760725.576523:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.576534:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.576656:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.576667:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54a56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19664896->19668991] (req 19664896->19668991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760725.576856:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508293888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60966us (61259us total) trans 4294972121 rc 0/0 00000100:00100000:0.0:1789760725.576869:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4833 00000100:00100000:0.0:1789760725.576882:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704557989118 time 704620580338 state 0x2 ref 0 00000100:00100000:1.0:1789760725.588126:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760725.590170:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508294016 00000100:00100000:2.0:1789760725.590201:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508294016 00000100:00100000:2.0:1789760725.591206:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704557989118 time 704620580338 state 0x0 ref 1 00000100:00100000:2.0:1789760725.591233:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704620840754 time 704620580338 state 0x1 ref 1 00000100:00100000:3.0:1789760725.592091:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704620840754 time 704696793307 state 0x0 ref 1 00000100:00100000:3.0:1789760725.592137:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508294016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760725.592224:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19668992, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:3.0:1789760725.592236:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.592258:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760725.592263:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.592270:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.592334:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.637338:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972122 is committed 00002000:00010000:3.0:1789760725.637459:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19668992 -> 19673088 00010000:00010000:3.0:1789760725.637480:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.637494:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.637504:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.637511:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760725.637516:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.637527:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.637630:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.637639:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19668992->19673087] (req 19668992->19673087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.637792:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508294016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45666us (49685us total) trans 4294972122 rc 0/0 00000100:00100000:3.0:1789760725.637805:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4834 00000100:00100000:3.0:1789760725.637820:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704620840754 time 704696793307 state 0x2 ref 0 00000100:00100000:2.0:1789760725.641172:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760725.641273:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508294144 00000100:00100000:3.0:1789760725.641320:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508294144 00000100:00100000:3.0:1789760725.641390:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704620840754 time 704696793307 state 0x0 ref 1 00000100:00100000:3.0:1789760725.641408:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704697053723 time 704696793307 state 0x1 ref 1 00000100:00100000:3.0:1789760725.641444:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704697053723 time 704746154555 state 0x0 ref 1 00000100:00100000:3.0:1789760725.641459:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508294144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760725.641530:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19673088, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:3.0:1789760725.641552:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.641574:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760725.641579:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.641587:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.641603:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760725.670859:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972123 is committed 00002000:00010000:3.0:1789760725.670967:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19673088 -> 19677184 00010000:00010000:3.0:1789760725.670987:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.671003:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.671013:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.671022:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760725.671027:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.671038:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.671065:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.671074:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19673088->19677183] (req 19673088->19677183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.671366:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508294144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29903us (30202us total) trans 4294972123 rc 0/0 00000100:00100000:3.0:1789760725.671378:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4835 00000100:00100000:3.0:1789760725.671390:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704697053723 time 704746154555 state 0x2 ref 0 00000100:00100000:1.0:1789760725.675013:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760725.675185:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508294272 00000100:00100000:0.0:1789760725.675208:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508294272 00000100:00100000:0.0:1789760725.675281:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704697053723 time 704746154555 state 0x0 ref 1 00000100:00100000:0.0:1789760725.675471:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704746414971 time 704746154555 state 0x1 ref 1 00000100:00100000:0.0:1789760725.675495:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704746414971 time 704780206130 state 0x0 ref 1 00000100:00100000:0.0:1789760725.675509:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508294272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760725.675683:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19677184, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:0.0:1789760725.675695:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.675714:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760725.675720:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.675727:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.675744:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760725.713935:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19677184 -> 19681280 00010000:00010000:1.0:1789760725.713966:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.713982:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a6b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.713993:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.714002:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760725.714009:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.714020:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a6b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.714047:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a6b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760725.714056:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54a6b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19677184->19681279] (req 19677184->19681279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760725.714254:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508294272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38740us (39246us total) trans 4294972124 rc 0/0 00000100:00100000:1.0:1789760725.714267:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4836 00000100:00100000:1.0:1789760725.714279:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704746414971 time 704780206130 state 0x2 ref 0 00000001:00080000:3.0:1789760725.714416:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972124 is committed 00000100:00100000:1.0:1789760725.717645:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760725.717909:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508294400 00000100:00100000:3.0:1789760725.717923:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508294400 00000100:00100000:3.0:1789760725.717996:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704746414971 time 704780206130 state 0x0 ref 1 00000100:00100000:3.0:1789760725.718017:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704780466546 time 704780206130 state 0x1 ref 1 00000100:00100000:3.0:1789760725.718082:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704780466546 time 704822791069 state 0x0 ref 1 00000100:00100000:3.0:1789760725.718097:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508294400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760725.718237:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19681280, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:3.0:1789760725.718249:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.718270:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760725.718275:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.718311:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.718332:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.758477:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760725.762226:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972125 is committed 00002000:00010000:3.0:1789760725.762356:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19681280 -> 19685376 00010000:00010000:3.0:1789760725.762377:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.762392:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a72 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.762402:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.762410:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760725.762415:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.762424:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a72 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.762450:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a72 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.762458:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a72 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19681280->19685375] (req 19681280->19685375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.762703:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508294400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44601us (45067us total) trans 4294972125 rc 0/0 00000100:00100000:3.0:1789760725.762716:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4837 00000100:00100000:3.0:1789760725.762730:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704780466546 time 704822791069 state 0x2 ref 0 00000100:00100000:2.0:1789760725.765711:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760725.765819:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508294528 00000100:00100000:3.0:1789760725.765833:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508294528 00000100:00100000:3.0:1789760725.765913:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704780466546 time 704822791069 state 0x0 ref 1 00000100:00100000:3.0:1789760725.765934:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704823051485 time 704822791069 state 0x1 ref 1 00000100:00100000:3.0:1789760725.766022:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704823051485 time 704870731455 state 0x0 ref 1 00000100:00100000:3.0:1789760725.766038:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508294528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760725.766121:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19685376, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:3.0:1789760725.766133:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.766155:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760725.766161:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.766170:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.766200:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760725.808913:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19685376 -> 19689472 00010000:00010000:3.0:1789760725.808939:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.808955:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.808965:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.808972:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760725.808977:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.808988:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.809012:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.809020:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19685376->19689471] (req 19685376->19689471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.809865:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972126 is committed 00000100:00100000:3.0:1789760725.811201:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508294528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45147us (45485us total) trans 4294972126 rc 0/0 00000100:00100000:3.0:1789760725.811220:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4838 00000100:00100000:3.0:1789760725.811232:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704823051485 time 704870731455 state 0x2 ref 0 00000100:00100000:1.0:1789760725.814245:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760725.814459:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508294656 00000100:00100000:0.0:1789760725.814474:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508294656 00000100:00100000:0.0:1789760725.814619:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704823051485 time 704870731455 state 0x0 ref 1 00000100:00100000:0.0:1789760725.814640:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704870991871 time 704870731455 state 0x1 ref 1 00000100:00100000:0.0:1789760725.814710:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704870991871 time 704919419353 state 0x0 ref 1 00000100:00100000:0.0:1789760725.814725:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508294656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760725.814792:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19689472, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:0.0:1789760725.814803:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54a80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.814824:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:0.0:1789760725.814830:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760725.814838:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54a80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760725.814855:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54a80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.854364:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972127 is committed 00002000:00010000:3.0:1789760725.854670:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19689472 -> 19693568 00010000:00010000:3.0:1789760725.854692:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54a80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.854707:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54a80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.854717:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54a80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.854725:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760725.854730:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54a80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.854741:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54a80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.854768:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54a80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.854776:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54a80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19689472->19693567] (req 19689472->19693567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.854909:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508294656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40179us (40671us total) trans 4294972127 rc 0/0 00000100:00100000:3.0:1789760725.854922:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4839 00000100:00100000:3.0:1789760725.854936:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704870991871 time 704919419353 state 0x2 ref 0 00000100:00100000:0.0:1789760725.858633:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760725.859059:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508294784 00000100:00100000:3.0:1789760725.859074:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508294784 00000100:00100000:3.0:1789760725.859226:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704870991871 time 704919419353 state 0x0 ref 1 00000100:00100000:3.0:1789760725.859245:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704919679769 time 704919419353 state 0x1 ref 1 00000100:00100000:3.0:1789760725.859387:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704919679769 time 704964096804 state 0x0 ref 1 00000100:00100000:3.0:1789760725.859403:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508294784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760725.859464:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19693568, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:3.0:1789760725.859474:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a87 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.859494:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760725.859499:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.859507:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.859524:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.886676:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b645180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3473:1876699534061696:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.887024:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.887139:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b647100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534062208:0@lo:400:kworker.0 00000100:00100000:2.0:1789760725.887169:0:6459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534061696 00000100:00100000:2.0:1789760725.887182:0:6459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534061696 00000100:00100000:3.0:1789760725.887185:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.887222:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b644e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534062720:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.887259:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.887282:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499bfece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534063232:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.887339:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.887373:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:3.0:1789760725.887381:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3473:1876699534062080:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.887409:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.887431:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534062592:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.887453:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.887471:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534063104:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.887497:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.887534:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534061824:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.887562:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.887585:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b645880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534062336:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.887613:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760725.887615:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534062080 00000100:00100000:0.0:1789760725.887622:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534062080 00000100:00100000:3.0:1789760725.887636:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b645500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534062848:0@lo:400:kworker.0 00000100:00100000:0.0:1789760725.887661:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534061824 00000100:00100000:0.0:1789760725.887666:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534061824 00000100:00100000:3.0:1789760725.887667:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.887690:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0498546b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534063360:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760725.887692:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534062336 00000100:00100000:0.0:1789760725.887697:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534062336 00000100:00100000:3.0:1789760725.887725:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760725.887728:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534062848 00000100:00100000:0.0:1789760725.887733:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534062848 00000100:00100000:0.0:1789760725.887756:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 863 00000100:00100000:3.0:1789760725.887756:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1789760725.887767:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b645c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534061952:0@lo:400:kworker.0 00000100:00100000:0.0:1789760725.887773:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3473:x1876699534062080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.887796:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.887818:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b647800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534062464:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.887851:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760725.887856:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3473:x1876699534062080:12345-0@lo:400:kworker.0 Request processed in 85us (442us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760725.887867:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 863 00000100:00100000:3.0:1789760725.887869:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b644a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534062976:0@lo:400:kworker.0 00000100:00100000:0.0:1789760725.887886:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534062464 00000100:00100000:0.0:1789760725.887891:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534062464 00000100:00100000:0.0:1789760725.887911:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 864 00000100:00100000:0.0:1789760725.887919:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfefb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534061824:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.887930:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760725.887959:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfefb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534061824:12345-0@lo:400:kworker.0 Request processed in 40us (395us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760725.887965:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 864 00000100:00100000:0.0:1789760725.887977:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534062976 00000100:00100000:0.0:1789760725.887981:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534062976 00000100:00100000:3.0:1789760725.887983:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534062208 00000100:00100000:3.0:1789760725.887989:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534062208 00000100:00100000:0.0:1789760725.887998:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 865 00000100:00100000:0.0:1789760725.888006:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534062336:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888033:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534061952 00000100:00100000:0.0:1789760725.888036:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfed180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534062336:12345-0@lo:400:kworker.0 Request processed in 31us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760725.888039:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534061952 00000100:00100000:0.0:1789760725.888048:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 865 00000100:00100000:3.0:1789760725.888065:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 418 00000100:00100000:0.0:1789760725.888070:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 866 00000100:00100000:3.0:1789760725.888079:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfeea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534062208:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760725.888083:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534062848:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888137:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfeea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534062208:12345-0@lo:400:kworker.0 Request processed in 61us (950us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760725.888141:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534062848:12345-0@lo:400:kworker.0 Request processed in 61us (472us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760725.888150:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 418 00000100:00100000:0.0:1789760725.888155:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 866 00000100:00100000:3.0:1789760725.888173:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 419 00000100:00100000:0.0:1789760725.888177:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 867 00000100:00100000:3.0:1789760725.888186:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987257800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534061952:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760725.888205:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534062464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888244:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987257800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534061952:12345-0@lo:400:kworker.0 Request processed in 60us (446us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760725.888252:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534062464:12345-0@lo:400:kworker.0 Request processed in 66us (400us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760725.888254:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 419 00000100:00100000:0.0:1789760725.888264:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 867 00000100:00100000:0.0:1789760725.888281:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 868 00000100:00100000:3.0:1789760725.888311:0:8262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534062720 00000100:00100000:0.0:1789760725.888316:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987257b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534062976:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888318:0:8262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534062720 00000100:00100000:3.0:1789760725.888353:0:8262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534063232 00000100:00100000:0.0:1789760725.888358:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987257b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534062976:12345-0@lo:400:kworker.0 Request processed in 45us (429us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760725.888359:0:8262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534063232 00000100:00100000:0.0:1789760725.888368:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 868 00000100:00100000:3.0:1789760725.888379:0:8262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534062592 00000100:00100000:3.0:1789760725.888383:0:8262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534062592 00000100:00100000:0.0:1789760725.888395:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534063104 00000100:00100000:0.0:1789760725.888403:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534063104 00000100:00100000:3.0:1789760725.888406:0:8262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:3.0:1789760725.888416:0:8262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534062720:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760725.888432:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:0.0:1789760725.888442:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534063232:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888459:0:8262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfedf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534062720:12345-0@lo:400:kworker.0 Request processed in 44us (1199us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760725.888467:0:8262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00100000:3.0:1789760725.888488:0:8262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:0.0:1789760725.888489:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfed880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534063232:12345-0@lo:400:kworker.0 Request processed in 45us (1145us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760725.888499:0:8262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfeed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534062592:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760725.888500:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00100000:0.0:1789760725.888519:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:0.0:1789760725.888529:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534063104:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888544:0:8262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfeed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534062592:12345-0@lo:400:kworker.0 Request processed in 47us (1089us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760725.888553:0:8262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:0.0:1789760725.888571:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfeca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534063104:12345-0@lo:400:kworker.0 Request processed in 44us (1071us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760725.888577:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:3.0:1789760725.888603:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534063360 00000100:00100000:3.0:1789760725.888608:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534063360 00000100:00080000:0.0:1789760725.888619:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760725.888626:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789760725.888629:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789760725.888631:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760725.888645:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:3.0:1789760725.888654:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534063360:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760725.888684:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3473:1876699534062080:0@lo:400:kworker.0 00002000:00000024:3.0:1789760725.888690:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760725.888705:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534062592:0@lo:400:kworker.0 00000100:00100000:0.0:1789760725.888725:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534063104:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888734:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534063360:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (1008us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760725.888743:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b647100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534062208:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888744:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:0.0:1789760725.888766:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b644e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534062720:0@lo:400:kworker.0 00000100:00100000:0.0:1789760725.888784:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499bfece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534063232:0@lo:400:kworker.0 00000100:00100000:2.0:1789760725.888788:0:6459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 466 00000100:00100000:3.0:1789760725.888797:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b645c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534061952:0@lo:400:kworker.0 00000100:00100000:2.0:1789760725.888804:0:6459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04989030450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3473:x1876699534061696:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888813:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b647800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534062464:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888829:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b644a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534062976:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888842:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534061824:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888858:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b645880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534062336:0@lo:400:kworker.0 00000100:00100000:2.0:1789760725.888862:0:6459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04989030450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3473:x1876699534061696:12345-0@lo:400:kworker.0 Request processed in 64us (2076us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760725.888868:0:6459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 466 00000100:00100000:0.0:1789760725.888872:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b645180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3473:1876699534061696:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888878:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b645500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534062848:0@lo:400:kworker.0 00000100:00100000:3.0:1789760725.888898:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0498546b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534063360:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760725.891916:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534063616:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760725.892089:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.894661:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534063488:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760725.894843:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760725.895481:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760725.895493:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04985468000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534063744:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760725.895538:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760725.897069:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534063616 00000100:00100000:1.0:1789760725.897088:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534063616 00000100:00100000:1.0:1789760725.897173:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534063488 00000100:00100000:1.0:1789760725.897177:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534063488 00000100:00100000:1.0:1789760725.897214:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 357 00000100:00100000:1.0:1789760725.897226:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987257480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534063616:12345-0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760725.897455:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 358 00000100:00100000:0.0:1789760725.897466:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534063488:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760725.897495:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3838836736 avail 3819962368 00002000:00000024:1.0:1789760725.897716:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760725.898002:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987257480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534063616:12345-0@lo:13:osp-pre-1-1.0 Request processed in 776us (5912us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760725.898013:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 357 00000100:00100000:1.0:1789760725.898086:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534063744 00000100:00100000:1.0:1789760725.898092:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534063744 00000100:00100000:1.0:1789760725.898122:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 359 00000100:00100000:1.0:1789760725.898131:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534063744:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760725.898138:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534063616:0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760725.898145:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3838836736 avail 3819962368 00000100:00100000:1.0:1789760725.898179:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534063744:12345-0@lo:13:osp-pre-0-0.0 Request processed in 49us (2641us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760725.898186:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 359 00000100:00100000:3.0:1789760725.898268:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04985468000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534063744:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760725.898476:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534063488:12345-0@lo:13:osp-pre-0-1.0 Request processed in 1012us (3632us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760725.898484:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 358 00000001:00080000:0.0:1789760725.898784:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972128 is committed 00002000:00010000:3.0:1789760725.898973:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19693568 -> 19697664 00010000:00010000:3.0:1789760725.898989:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.899004:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.899014:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.899021:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760725.899123:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.899142:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.899170:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.899179:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19693568->19697663] (req 19693568->19697663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.899327:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508294784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39879us (40660us total) trans 4294972128 rc 0/0 00000100:00100000:3.0:1789760725.899340:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4840 00000100:00100000:3.0:1789760725.899351:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704919679769 time 704964096804 state 0x2 ref 0 00000100:00100000:2.0:1789760725.899950:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534063488:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760725.902763:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760725.904956:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508294912 00000100:00100000:3.0:1789760725.904979:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508294912 00000100:00100000:3.0:1789760725.905079:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704919679769 time 704964096804 state 0x0 ref 1 00000100:00100000:3.0:1789760725.905100:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704964357220 time 704964096804 state 0x1 ref 1 00000100:00100000:3.0:1789760725.905180:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704964357220 time 705009888685 state 0x0 ref 1 00000100:00100000:3.0:1789760725.905208:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508294912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760725.905273:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19697664, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:3.0:1789760725.905307:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.905327:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760725.905333:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.905340:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.905356:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760725.935109:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972129 is committed 00002000:00010000:3.0:1789760725.935222:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19697664 -> 19701760 00010000:00010000:3.0:1789760725.935243:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.935257:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.935268:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.935276:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760725.935281:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.935317:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.935343:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.935353:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19697664->19701759] (req 19697664->19701759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.935520:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508294912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30319us (32764us total) trans 4294972129 rc 0/0 00000100:00100000:3.0:1789760725.935533:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4841 00000100:00100000:3.0:1789760725.935622:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704964357220 time 705009888685 state 0x2 ref 0 00000100:00100000:1.0:1789760725.938416:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760725.938539:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508295040 00000100:00100000:3.0:1789760725.938556:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508295040 00000100:00100000:3.0:1789760725.938907:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 704964357220 time 705009888685 state 0x0 ref 1 00000100:00100000:3.0:1789760725.938931:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705010149101 time 705009888685 state 0x1 ref 1 00000100:00100000:3.0:1789760725.939019:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705010149101 time 705043728546 state 0x0 ref 1 00000100:00100000:3.0:1789760725.939036:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508295040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760725.939170:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19701760, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:3.0:1789760725.939184:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.939221:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760725.939227:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.939236:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.939260:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760725.991204:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972130 is committed 00002000:00010000:3.0:1789760725.991684:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19701760 -> 19705856 00010000:00010000:3.0:1789760725.991708:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.991728:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.991738:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.991807:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760725.991813:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.991825:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.991851:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.991859:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54a95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19701760->19705855] (req 19701760->19705855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760725.992096:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508295040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52998us (53633us total) trans 4294972130 rc 0/0 00000100:00100000:3.0:1789760725.992126:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4842 00000100:00100000:3.0:1789760725.992137:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705010149101 time 705043728546 state 0x2 ref 0 00000100:00100000:0.0:1789760725.996902:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760725.997002:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508295168 00000100:00100000:3.0:1789760725.997017:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508295168 00000100:00100000:3.0:1789760725.997087:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705010149101 time 705043728546 state 0x0 ref 1 00000100:00100000:3.0:1789760725.997159:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705043988962 time 705043728546 state 0x1 ref 1 00000100:00100000:3.0:1789760725.997244:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705043988962 time 705101952491 state 0x0 ref 1 00000100:00100000:3.0:1789760725.997394:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508295168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760725.997527:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19705856, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:3.0:1789760725.997590:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.997612:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760725.997616:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760725.997621:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760725.997637:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.049066:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972131 is committed 00002000:00010000:2.0:1789760726.051627:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19705856 -> 19709952 00010000:00010000:2.0:1789760726.051656:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.051672:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.051683:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.051698:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760726.051704:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.051716:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.051743:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.051752:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54a9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19705856->19709951] (req 19705856->19709951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760726.051897:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508295168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54624us (55003us total) trans 4294972131 rc 0/0 00000100:00100000:2.0:1789760726.051910:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4843 00000100:00100000:2.0:1789760726.051923:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705043988962 time 705101952491 state 0x2 ref 0 00000100:00100000:3.0:1789760726.054747:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760726.054906:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508295296 00000100:00100000:2.0:1789760726.054919:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508295296 00000100:00100000:2.0:1789760726.054989:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705043988962 time 705101952491 state 0x0 ref 1 00000100:00100000:2.0:1789760726.055007:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705102212907 time 705101952491 state 0x1 ref 1 00000100:00100000:2.0:1789760726.055043:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705102212907 time 705159753235 state 0x0 ref 1 00000100:00100000:2.0:1789760726.055059:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508295296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760726.055124:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19709952, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:2.0:1789760726.055135:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54aa3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.055154:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:2.0:1789760726.055159:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760726.055167:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54aa3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760726.055183:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54aa3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.104433:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972132 is committed 00002000:00010000:3.0:1789760726.104654:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19709952 -> 19714048 00010000:00010000:3.0:1789760726.104674:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54aa3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.104689:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54aa3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.104698:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54aa3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.104706:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760726.104711:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54aa3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.104721:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54aa3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.104748:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54aa3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.104757:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54aa3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19709952->19714047] (req 19709952->19714047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.104883:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508295296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49819us (50144us total) trans 4294972132 rc 0/0 00000100:00100000:3.0:1789760726.104896:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4844 00000100:00100000:3.0:1789760726.104907:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705102212907 time 705159753235 state 0x2 ref 0 00000100:00100000:3.0:1789760726.107490:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760726.107626:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508295424 00000100:00100000:0.0:1789760726.107642:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508295424 00000100:00100000:0.0:1789760726.107718:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705102212907 time 705159753235 state 0x0 ref 1 00000100:00100000:0.0:1789760726.107739:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705160013651 time 705159753235 state 0x1 ref 1 00000100:00100000:0.0:1789760726.107788:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705160013651 time 705212498039 state 0x0 ref 1 00000100:00100000:0.0:1789760726.107803:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508295424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760726.107873:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19714048, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:0.0:1789760726.107884:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54aaa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.107903:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.107919:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.107926:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54aaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.107943:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54aaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.151811:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972133 is committed 00002000:00010000:3.0:1789760726.152003:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19714048 -> 19718144 00010000:00010000:3.0:1789760726.152022:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54aaa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.152038:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54aaa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.152048:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54aaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.152058:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760726.152064:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54aaa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.152075:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54aaa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.152111:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54aaa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.152120:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54aaa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19714048->19718143] (req 19714048->19718143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.152258:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508295424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44449us (44774us total) trans 4294972133 rc 0/0 00000100:00100000:3.0:1789760726.152270:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4845 00000100:00100000:3.0:1789760726.152320:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705160013651 time 705212498039 state 0x2 ref 0 00000100:00100000:0.0:1789760726.158854:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760726.159739:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508295552 00000100:00100000:2.0:1789760726.159868:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508295552 00000100:00100000:3.0:1789760726.160118:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705160013651 time 705212498039 state 0x0 ref 1 00000100:00100000:3.0:1789760726.160138:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705212758455 time 705212498039 state 0x1 ref 1 00000100:00100000:3.0:1789760726.160233:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705212758455 time 705264941413 state 0x0 ref 1 00000100:00100000:3.0:1789760726.160249:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508295552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760726.160353:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19718144, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:3.0:1789760726.160365:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54ab1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.160384:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760726.160390:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.160398:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54ab1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.160415:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54ab1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.194567:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972134 is committed 00002000:00010000:1.0:1789760726.194797:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19718144 -> 19722240 00010000:00010000:1.0:1789760726.194817:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54ab1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.194833:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54ab1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.194843:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54ab1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.194851:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760726.194857:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54ab1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.194868:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54ab1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.194894:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54ab1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.194903:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54ab1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19718144->19722239] (req 19718144->19722239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.196059:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508295552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35711us (37117us total) trans 4294972134 rc 0/0 00000100:00100000:1.0:1789760726.196074:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4846 00000100:00100000:1.0:1789760726.196087:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705212758455 time 705264941413 state 0x2 ref 0 00000100:00100000:1.0:1789760726.199523:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760726.199940:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508295680 00000100:00100000:3.0:1789760726.199956:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508295680 00000100:00100000:3.0:1789760726.200034:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705212758455 time 705264941413 state 0x0 ref 1 00000100:00100000:3.0:1789760726.200056:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705265201829 time 705264941413 state 0x1 ref 1 00000100:00100000:3.0:1789760726.200096:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705265201829 time 705304805111 state 0x0 ref 1 00000100:00100000:3.0:1789760726.200122:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508295680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760726.200200:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19722240, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:3.0:1789760726.200212:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54ab8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.200232:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760726.200237:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760726.200245:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54ab8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.200263:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54ab8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760726.235929:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19722240 -> 19726336 00010000:00010000:0.0:1789760726.235954:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54ab8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.235969:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54ab8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.235979:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54ab8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.235992:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760726.235998:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54ab8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.236009:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54ab8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.236035:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54ab8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.236043:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54ab8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19722240->19726335] (req 19722240->19726335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.236176:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508295680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36050us (36662us total) trans 4294972135 rc 0/0 00000100:00100000:0.0:1789760726.236197:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4847 00000100:00100000:0.0:1789760726.236210:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705265201829 time 705304805111 state 0x2 ref 0 00000100:00100000:3.0:1789760726.239794:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760726.239880:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508295808 00000100:00100000:0.0:1789760726.239892:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508295808 00000100:00100000:0.0:1789760726.239963:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705265201829 time 705304805111 state 0x0 ref 1 00000100:00100000:0.0:1789760726.239983:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705305065527 time 705304805111 state 0x1 ref 1 00000100:00100000:0.0:1789760726.240020:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705305065527 time 705344729854 state 0x0 ref 1 00000100:00100000:0.0:1789760726.240037:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508295808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760726.240099:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19726336, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:0.0:1789760726.240110:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54abf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.240129:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.240134:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.240142:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54abf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.240159:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54abf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.241871:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972135 is committed 00000001:00080000:2.0:1789760726.266393:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972136 is committed 00002000:00010000:1.0:1789760726.266622:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19726336 -> 19730432 00010000:00010000:1.0:1789760726.266642:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54abf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.266657:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54abf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.266668:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54abf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.266675:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760726.266682:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54abf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.266692:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54abf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.266717:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54abf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.266725:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54abf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19726336->19730431] (req 19726336->19730431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.266890:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508295808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26848us (27104us total) trans 4294972136 rc 0/0 00000100:00100000:1.0:1789760726.266903:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4848 00000100:00100000:1.0:1789760726.266915:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705305065527 time 705344729854 state 0x2 ref 0 00000100:00100000:2.0:1789760726.269821:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760726.270394:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508295936 00000100:00100000:1.0:1789760726.270407:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508295936 00000100:00100000:1.0:1789760726.270485:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705305065527 time 705344729854 state 0x0 ref 1 00000100:00100000:1.0:1789760726.270505:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705344990270 time 705344729854 state 0x1 ref 1 00000100:00100000:1.0:1789760726.270662:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705344990270 time 705375371359 state 0x0 ref 1 00000100:00100000:1.0:1789760726.270678:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508295936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760726.270751:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19730432, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:1.0:1789760726.270763:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54ac6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.270786:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760726.270792:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.270799:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54ac6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.270817:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54ac6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.303087:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972137 is committed 00002000:00010000:1.0:1789760726.303221:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19730432 -> 19734528 00010000:00010000:1.0:1789760726.303241:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54ac6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.303263:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54ac6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.303274:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54ac6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.303282:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760726.303323:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54ac6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.303335:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54ac6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.303361:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54ac6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.303369:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54ac6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19730432->19734527] (req 19730432->19734527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.303576:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508295936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32893us (33764us total) trans 4294972137 rc 0/0 00000100:00100000:1.0:1789760726.303588:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4849 00000100:00100000:1.0:1789760726.303601:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705344990270 time 705375371359 state 0x2 ref 0 00000100:00100000:3.0:1789760726.307532:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760726.307668:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508296064 00000100:00100000:0.0:1789760726.307682:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508296064 00000100:00100000:0.0:1789760726.307757:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705344990270 time 705375371359 state 0x0 ref 1 00000100:00100000:0.0:1789760726.307779:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705375631775 time 705375371359 state 0x1 ref 1 00000100:00100000:0.0:1789760726.307814:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705375631775 time 705412523995 state 0x0 ref 1 00000100:00100000:0.0:1789760726.307829:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508296064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760726.307899:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19734528, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:0.0:1789760726.307911:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54acd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.307937:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.307942:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.307950:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54acd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.307966:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54acd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.337864:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972138 is committed 00002000:00010000:0.0:1789760726.337986:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19734528 -> 19738624 00010000:00010000:0.0:1789760726.338005:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54acd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.338021:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54acd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.338030:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54acd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.338038:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760726.338044:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54acd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.338055:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54acd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.338079:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54acd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.338087:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54acd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19734528->19738623] (req 19734528->19738623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.338410:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508296064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30576us (30885us total) trans 4294972138 rc 0/0 00000100:00100000:0.0:1789760726.338421:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4850 00000100:00100000:0.0:1789760726.338433:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705375631775 time 705412523995 state 0x2 ref 0 00000100:00100000:2.0:1789760726.342686:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760726.342821:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508296192 00000100:00100000:0.0:1789760726.342834:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508296192 00000100:00100000:0.0:1789760726.342907:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705375631775 time 705412523995 state 0x0 ref 1 00000100:00100000:0.0:1789760726.342926:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705412784411 time 705412523995 state 0x1 ref 1 00000100:00100000:0.0:1789760726.343005:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705412784411 time 705447714289 state 0x0 ref 1 00000100:00100000:0.0:1789760726.343020:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508296192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760726.343085:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19738624, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:0.0:1789760726.343096:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ad4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.343194:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.343200:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.343208:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ad4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.343231:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ad4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.371236:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972139 is committed 00002000:00010000:3.0:1789760726.371407:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19738624 -> 19742720 00010000:00010000:3.0:1789760726.371427:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ad4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.371444:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ad4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.371451:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ad4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.371456:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760726.371460:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ad4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.371466:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ad4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.371487:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ad4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.371493:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ad4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19738624->19742719] (req 19738624->19742719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.371616:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508296192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28589us (28937us total) trans 4294972139 rc 0/0 00000100:00100000:3.0:1789760726.371628:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4851 00000100:00100000:3.0:1789760726.371641:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705412784411 time 705447714289 state 0x2 ref 0 00000100:00100000:2.0:1789760726.375090:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760726.375428:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508296320 00000100:00100000:1.0:1789760726.375441:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508296320 00000100:00100000:1.0:1789760726.375522:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705412784411 time 705447714289 state 0x0 ref 1 00000100:00100000:1.0:1789760726.375612:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705447974705 time 705447714289 state 0x1 ref 1 00000100:00100000:1.0:1789760726.375689:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705447974705 time 705480398784 state 0x0 ref 1 00000100:00100000:1.0:1789760726.375704:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508296320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760726.375778:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19742720, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:1.0:1789760726.375790:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54adb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.375812:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760726.375818:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.375826:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54adb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.375844:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54adb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.430236:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972140 is committed 00002000:00010000:3.0:1789760726.430370:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19742720 -> 19746816 00010000:00010000:3.0:1789760726.430391:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54adb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.430406:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54adb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.430416:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54adb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.430424:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760726.430430:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54adb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.430440:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54adb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.430466:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54adb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.430474:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54adb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19742720->19746815] (req 19742720->19746815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.430642:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508296320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54934us (55560us total) trans 4294972140 rc 0/0 00000100:00100000:3.0:1789760726.430655:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4852 00000100:00100000:3.0:1789760726.430668:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705447974705 time 705480398784 state 0x2 ref 0 00000100:00100000:2.0:1789760726.438333:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760726.438491:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508296448 00000100:00100000:1.0:1789760726.438504:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508296448 00000100:00100000:1.0:1789760726.438580:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705447974705 time 705480398784 state 0x0 ref 1 00000100:00100000:1.0:1789760726.438600:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705480659200 time 705480398784 state 0x1 ref 1 00000100:00100000:1.0:1789760726.438687:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705480659200 time 705543396145 state 0x0 ref 1 00000100:00100000:1.0:1789760726.438706:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508296448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760726.438842:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19746816, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:1.0:1789760726.438854:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54ae2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.438873:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760726.438878:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.438885:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54ae2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.438901:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54ae2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760726.504933:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972141 is committed 00002000:00010000:3.0:1789760726.505080:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19746816 -> 19750912 00010000:00010000:3.0:1789760726.505099:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54ae2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.505115:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54ae2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.505125:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54ae2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.505133:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760726.505138:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54ae2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.505157:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54ae2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.505182:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54ae2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.505199:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe54ae2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19746816->19750911] (req 19746816->19750911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.505369:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508296448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 66659us (67086us total) trans 4294972141 rc 0/0 00000100:00100000:3.0:1789760726.505380:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4853 00000100:00100000:3.0:1789760726.505392:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705480659200 time 705543396145 state 0x2 ref 0 00000100:00100000:3.0:1789760726.516997:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760726.517110:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508296576 00000100:00100000:0.0:1789760726.517124:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508296576 00000100:00100000:0.0:1789760726.517206:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705480659200 time 705543396145 state 0x0 ref 1 00000100:00100000:0.0:1789760726.517226:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705543656561 time 705543396145 state 0x1 ref 1 00000100:00100000:0.0:1789760726.517263:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705543656561 time 705621972419 state 0x0 ref 1 00000100:00100000:0.0:1789760726.517277:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508296576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760726.517363:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19750912, mtime: 0x6aad94d6, atime: 0x0, ctime: 0x6aad94d6, blocks: 0x9807 00010000:00010000:0.0:1789760726.517374:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ae9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.517393:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.517399:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.517406:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ae9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.517423:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ae9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760726.572459:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972142 is committed 00002000:00010000:3.0:1789760726.572697:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19750912 -> 19755008 00002000:00010000:3.0:1789760726.572702:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760726 -> 1789760727 00002000:00010000:3.0:1789760726.572705:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760726 -> 1789760727 00010000:00010000:3.0:1789760726.572723:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ae9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.572739:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ae9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.572748:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ae9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.572756:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760726.572762:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ae9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.572771:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ae9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.572798:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ae9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.572806:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ae9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19750912->19755007] (req 19750912->19755007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.572973:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508296576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55691us (55984us total) trans 4294972142 rc 0/0 00000100:00100000:3.0:1789760726.572987:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4854 00000100:00100000:3.0:1789760726.573001:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705543656561 time 705621972419 state 0x2 ref 0 00000100:00100000:3.0:1789760726.576544:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760726.576648:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508296704 00000100:00100000:1.0:1789760726.576663:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508296704 00000100:00100000:1.0:1789760726.576737:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705543656561 time 705621972419 state 0x0 ref 1 00000100:00100000:1.0:1789760726.576756:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705622232835 time 705621972419 state 0x1 ref 1 00000100:00100000:1.0:1789760726.576792:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705622232835 time 705681502307 state 0x0 ref 1 00000100:00100000:1.0:1789760726.576807:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508296704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760726.576872:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19755008, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:1.0:1789760726.576884:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.576905:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760726.576911:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.576918:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.576935:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760726.661124:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19755008 -> 19759104 00010000:00010000:1.0:1789760726.661151:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.661166:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.661176:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.661194:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760726.661199:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.661210:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.661236:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.661244:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19755008->19759103] (req 19755008->19759103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.661405:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508296704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 84594us (84881us total) trans 4294972143 rc 0/0 00000100:00100000:1.0:1789760726.661417:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4855 00000100:00100000:1.0:1789760726.661430:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705622232835 time 705681502307 state 0x2 ref 0 00000001:00080000:3.0:1789760726.662955:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972143 is committed 00000100:00100000:2.0:1789760726.671845:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760726.671962:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508296832 00000100:00100000:1.0:1789760726.671975:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508296832 00000100:00100000:1.0:1789760726.672052:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705622232835 time 705681502307 state 0x0 ref 1 00000100:00100000:1.0:1789760726.672072:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705681762723 time 705681502307 state 0x1 ref 1 00000100:00100000:1.0:1789760726.672112:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705681762723 time 705776822232 state 0x0 ref 1 00000100:00100000:1.0:1789760726.672125:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508296832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760726.672209:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19759104, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:1.0:1789760726.672221:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.672240:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760726.672246:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.672254:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.672277:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.735490:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972144 is committed 00002000:00010000:0.0:1789760726.735661:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19759104 -> 19763200 00010000:00010000:0.0:1789760726.735683:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.735697:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.735707:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.735715:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760726.735720:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.735730:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.735757:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.735764:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54af7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19759104->19763199] (req 19759104->19763199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.735889:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508296832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 63759us (64052us total) trans 4294972144 rc 0/0 00000100:00100000:0.0:1789760726.735901:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4856 00000100:00100000:0.0:1789760726.735913:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705681762723 time 705776822232 state 0x2 ref 0 00000100:00100000:3.0:1789760726.738858:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760726.739335:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508296960 00000100:00100000:0.0:1789760726.739349:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508296960 00000100:00100000:0.0:1789760726.739424:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705681762723 time 705776822232 state 0x0 ref 1 00000100:00100000:0.0:1789760726.739443:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705777082648 time 705776822232 state 0x1 ref 1 00000100:00100000:0.0:1789760726.739505:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705777082648 time 705844215699 state 0x0 ref 1 00000100:00100000:0.0:1789760726.739517:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508296960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760726.739593:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19763200, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:0.0:1789760726.739605:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54afe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.739625:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.739630:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.739638:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54afe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.739654:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54afe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760726.766958:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19763200 -> 19767296 00010000:00010000:1.0:1789760726.766986:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54afe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.767005:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54afe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.767016:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54afe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.767024:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760726.767030:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54afe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.767040:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54afe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.767077:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54afe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.767086:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54afe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19763200->19767295] (req 19763200->19767295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760726.767232:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508296960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27707us (28381us total) trans 4294972145 rc 0/0 00000100:00100000:1.0:1789760726.767246:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4857 00000100:00100000:1.0:1789760726.767259:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705777082648 time 705844215699 state 0x2 ref 0 00000001:00080000:2.0:1789760726.767503:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972145 is committed 00000100:00100000:3.0:1789760726.771269:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760726.771501:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508297088 00000100:00100000:1.0:1789760726.771516:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508297088 00000100:00100000:1.0:1789760726.771600:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705777082648 time 705844215699 state 0x0 ref 1 00000100:00100000:1.0:1789760726.771621:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705844476115 time 705844215699 state 0x1 ref 1 00000100:00100000:1.0:1789760726.771662:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705844476115 time 705876371564 state 0x0 ref 1 00000100:00100000:1.0:1789760726.771676:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508297088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760726.771739:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19767296, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:1.0:1789760726.771750:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.771769:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760726.771774:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.771782:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.771798:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.783163:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760726.829338:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19767296 -> 19771392 00010000:00010000:1.0:1789760726.829365:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.829380:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.829389:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.829398:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760726.829403:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.829486:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.829517:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.829525:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19767296->19771391] (req 19767296->19771391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760726.829871:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972146 is committed 00000100:00100000:1.0:1789760726.829949:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508297088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58259us (58677us total) trans 4294972146 rc 0/0 00000100:00100000:1.0:1789760726.829966:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4858 00000100:00100000:1.0:1789760726.829979:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705844476115 time 705876371564 state 0x2 ref 0 00000100:00100000:3.0:1789760726.833134:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760726.833785:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508297216 00000100:00100000:0.0:1789760726.833799:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508297216 00000100:00100000:0.0:1789760726.833875:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705844476115 time 705876371564 state 0x0 ref 1 00000100:00100000:0.0:1789760726.833895:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705876631980 time 705876371564 state 0x1 ref 1 00000100:00100000:0.0:1789760726.834162:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705876631980 time 705938715807 state 0x0 ref 1 00000100:00100000:0.0:1789760726.834179:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508297216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760726.834259:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19771392, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:0.0:1789760726.834271:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54b0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.834307:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:0.0:1789760726.834313:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760726.834321:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54b0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760726.834339:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54b0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760726.874282:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19771392 -> 19775488 00010000:00010000:3.0:1789760726.874337:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54b0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.874353:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54b0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.874364:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54b0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.874373:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760726.874380:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54b0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.874395:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54b0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.874422:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54b0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.874431:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54b0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19771392->19775487] (req 19771392->19775487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.874559:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508297216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40375us (41432us total) trans 4294972147 rc 0/0 00000100:00100000:3.0:1789760726.874571:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4859 00000100:00100000:3.0:1789760726.874583:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705876631980 time 705938715807 state 0x2 ref 0 00000001:00080000:1.0:1789760726.874946:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972147 is committed 00000100:00100000:2.0:1789760726.877726:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760726.878144:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508297344 00000100:00100000:1.0:1789760726.878223:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508297344 00000100:00100000:1.0:1789760726.878381:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705876631980 time 705938715807 state 0x0 ref 1 00000100:00100000:1.0:1789760726.878401:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705938976223 time 705938715807 state 0x1 ref 1 00000100:00100000:1.0:1789760726.878430:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705938976223 time 705983139562 state 0x0 ref 1 00000100:00100000:1.0:1789760726.878445:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508297344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760726.878514:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19775488, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:1.0:1789760726.878525:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.878675:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=705s v=(0 0) 00010000:00010000:1.0:1789760726.878680:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.878687:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.878704:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760726.916812:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c9c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534063872:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760726.916906:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760726.917022:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534063872 00000100:00100000:0.0:1789760726.917031:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534063872 00000100:00100000:0.0:1789760726.917103:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 420 00000100:00100000:0.0:1789760726.917113:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534063872:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760726.917162:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789760726.917226:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534063872:12345-0@lo:41:osp-pre-0-1.0 Request processed in 114us (320us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760726.917235:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 420 00000100:00100000:0.0:1789760726.917310:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c9c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534063872:0@lo:41:osp-pre-0-1.0 00000001:00080000:2.0:1789760726.937934:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972148 is committed 00002000:00010000:3.0:1789760726.938097:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19775488 -> 19779584 00010000:00010000:3.0:1789760726.938116:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.938131:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.938141:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.938149:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760726.938155:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.938165:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.938204:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760726.938213:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54b13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19775488->19779583] (req 19775488->19779583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760726.938383:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508297344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59935us (60663us total) trans 4294972148 rc 0/0 00000100:00100000:3.0:1789760726.938395:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4860 00000100:00100000:3.0:1789760726.938408:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705938976223 time 705983139562 state 0x2 ref 0 00000100:00100000:2.0:1789760726.941424:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760726.941713:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508297472 00000100:00100000:1.0:1789760726.941726:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508297472 00000100:00100000:1.0:1789760726.941802:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705938976223 time 705983139562 state 0x0 ref 1 00000100:00100000:1.0:1789760726.941821:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705983399978 time 705983139562 state 0x1 ref 1 00000100:00100000:1.0:1789760726.941855:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705983399978 time 706046565880 state 0x0 ref 1 00000100:00100000:1.0:1789760726.941869:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508297472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760726.941935:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19779584, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:1.0:1789760726.941946:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54b1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.941976:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760726.941981:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760726.941989:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54b1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760726.942013:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54b1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760727.028208:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19779584 -> 19783680 00010000:00010000:0.0:1789760727.028235:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54b1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.028251:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54b1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.028261:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54b1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.028270:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760727.028275:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54b1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.028305:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54b1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.028335:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54b1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.028345:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54b1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19779584->19783679] (req 19779584->19783679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.028474:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508297472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 86601us (87060us total) trans 4294972149 rc 0/0 00000100:00100000:0.0:1789760727.028486:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4861 00000100:00100000:0.0:1789760727.028500:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705983399978 time 706046565880 state 0x2 ref 0 00000001:00080000:3.0:1789760727.029120:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972149 is committed 00000100:00100000:3.0:1789760727.035437:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760727.035547:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508297600 00000100:00100000:0.0:1789760727.035560:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508297600 00000100:00100000:0.0:1789760727.035646:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 705983399978 time 706046565880 state 0x0 ref 1 00000100:00100000:0.0:1789760727.035664:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706046826296 time 706046565880 state 0x1 ref 1 00000100:00100000:0.0:1789760727.035694:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706046826296 time 706140405053 state 0x0 ref 1 00000100:00100000:0.0:1789760727.035706:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508297600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760727.035768:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19783680, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:0.0:1789760727.035778:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54b21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.035798:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760727.035804:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.035822:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54b21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.035839:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54b21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760727.036394:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789760727.079502:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19783680 -> 19787776 00010000:00010000:0.0:1789760727.079591:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54b21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.079613:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54b21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.079623:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54b21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.079632:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760727.079637:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54b21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.079755:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54b21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.079782:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54b21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.079790:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54b21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19783680->19787775] (req 19783680->19787775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760727.079982:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508297600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44270us (44553us total) trans 4294972150 rc 0/0 00000100:00100000:0.0:1789760727.079994:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4862 00000100:00100000:0.0:1789760727.080006:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706046826296 time 706140405053 state 0x2 ref 0 00000001:00080000:2.0:1789760727.080650:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972150 is committed 00000100:00100000:2.0:1789760727.083059:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760727.083339:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508297728 00000100:00100000:3.0:1789760727.083352:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508297728 00000100:00100000:3.0:1789760727.083426:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706046826296 time 706140405053 state 0x0 ref 1 00000100:00100000:3.0:1789760727.083446:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706140665469 time 706140405053 state 0x1 ref 1 00000100:00100000:3.0:1789760727.083479:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706140665469 time 706188189053 state 0x0 ref 1 00000100:00100000:3.0:1789760727.083495:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508297728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760727.083692:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19787776, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:3.0:1789760727.083704:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.083724:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760727.083729:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.083736:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.083754:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.118249:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972151 is committed 00002000:00010000:3.0:1789760727.118461:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19787776 -> 19791872 00010000:00010000:3.0:1789760727.118483:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.118498:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.118508:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.118517:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760727.118522:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.118605:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.118633:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.118641:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19787776->19791871] (req 19787776->19791871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.118923:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508297728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35425us (35870us total) trans 4294972151 rc 0/0 00000100:00100000:3.0:1789760727.118937:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4863 00000100:00100000:3.0:1789760727.118949:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706140665469 time 706188189053 state 0x2 ref 0 00000100:00100000:1.0:1789760727.130466:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760727.130684:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508297856 00000100:00100000:0.0:1789760727.130698:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508297856 00000100:00100000:0.0:1789760727.130772:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706140665469 time 706188189053 state 0x0 ref 1 00000100:00100000:0.0:1789760727.130793:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706188449469 time 706188189053 state 0x1 ref 1 00000100:00100000:0.0:1789760727.130829:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706188449469 time 706235538492 state 0x0 ref 1 00000100:00100000:0.0:1789760727.130844:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508297856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760727.130913:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19791872, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:0.0:1789760727.130925:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54b2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.130944:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:0.0:1789760727.130949:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760727.130957:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54b2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760727.130974:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54b2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760727.186040:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19791872 -> 19795968 00000001:00080000:0.0:1789760727.186062:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972152 is committed 00010000:00010000:3.0:1789760727.186068:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54b2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.186085:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54b2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.186096:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54b2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.186104:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760727.186109:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54b2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.186126:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54b2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.186151:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54b2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.186160:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54b2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19791872->19795967] (req 19791872->19795967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.186345:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508297856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55496us (55897us total) trans 4294972152 rc 0/0 00000100:00100000:3.0:1789760727.186357:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4864 00000100:00100000:3.0:1789760727.186369:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706188449469 time 706235538492 state 0x2 ref 0 00000100:00100000:0.0:1789760727.189754:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760727.190118:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508297984 00000100:00100000:2.0:1789760727.190133:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508297984 00000100:00100000:2.0:1789760727.190220:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706188449469 time 706235538492 state 0x0 ref 1 00000100:00100000:2.0:1789760727.190240:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706235798908 time 706235538492 state 0x1 ref 1 00000100:00100000:2.0:1789760727.190423:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706235798908 time 706294995758 state 0x0 ref 1 00000100:00100000:2.0:1789760727.190441:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508297984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760727.190509:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19795968, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:2.0:1789760727.190520:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54b36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.190541:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760727.190546:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.190553:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54b36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.190571:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54b36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.261347:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972153 is committed 00002000:00010000:3.0:1789760727.262196:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19795968 -> 19800064 00010000:00010000:3.0:1789760727.262221:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54b36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.262238:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54b36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.262247:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54b36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.262256:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760727.262262:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54b36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.262273:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54b36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.262328:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54b36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.262337:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54b36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19795968->19800063] (req 19795968->19800063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.262473:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508297984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 72030us (72728us total) trans 4294972153 rc 0/0 00000100:00100000:3.0:1789760727.262485:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4865 00000100:00100000:3.0:1789760727.262498:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706235798908 time 706294995758 state 0x2 ref 0 00000100:00100000:0.0:1789760727.269154:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760727.269378:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508298112 00000100:00100000:3.0:1789760727.269391:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508298112 00000100:00100000:3.0:1789760727.269468:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706235798908 time 706294995758 state 0x0 ref 1 00000100:00100000:3.0:1789760727.269489:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706295256174 time 706294995758 state 0x1 ref 1 00000100:00100000:3.0:1789760727.269540:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706295256174 time 706374239220 state 0x0 ref 1 00000100:00100000:3.0:1789760727.269555:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508298112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760727.269621:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19800064, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:3.0:1789760727.269634:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.269654:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760727.269659:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.269666:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.269683:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760727.312547:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19800064 -> 19804160 00010000:00010000:2.0:1789760727.312575:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.312592:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.312599:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.312605:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760727.312609:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.312616:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.312638:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.312645:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19800064->19804159] (req 19800064->19804159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.312897:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508298112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43336us (43747us total) trans 4294972154 rc 0/0 00000100:00100000:2.0:1789760727.312911:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4866 00000100:00100000:2.0:1789760727.312923:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706295256174 time 706374239220 state 0x2 ref 0 00000001:00080000:2.0:1789760727.312975:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972154 is committed 00000100:00100000:1.0:1789760727.315692:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760727.316091:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508298240 00000100:00100000:3.0:1789760727.316112:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508298240 00000100:00100000:3.0:1789760727.316198:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706295256174 time 706374239220 state 0x0 ref 1 00000100:00100000:3.0:1789760727.316218:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706374499636 time 706374239220 state 0x1 ref 1 00000100:00100000:3.0:1789760727.316312:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706374499636 time 706421020813 state 0x0 ref 1 00000100:00100000:3.0:1789760727.316328:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508298240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760727.316392:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19804160, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:3.0:1789760727.316404:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.316424:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760727.316430:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.316437:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.316455:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.367087:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972155 is committed 00002000:00010000:3.0:1789760727.367277:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19804160 -> 19808256 00010000:00010000:3.0:1789760727.367319:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.367335:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.367346:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.367353:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760727.367359:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.367370:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.367398:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.367408:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19804160->19808255] (req 19804160->19808255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.367977:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508298240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51643us (52290us total) trans 4294972155 rc 0/0 00000100:00100000:3.0:1789760727.367991:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4867 00000100:00100000:3.0:1789760727.368003:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706374499636 time 706421020813 state 0x2 ref 0 00000100:00100000:0.0:1789760727.372981:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760727.373268:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508298368 00000100:00100000:2.0:1789760727.373320:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508298368 00000100:00100000:2.0:1789760727.373398:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706374499636 time 706421020813 state 0x0 ref 1 00000100:00100000:2.0:1789760727.373417:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706421281229 time 706421020813 state 0x1 ref 1 00000100:00100000:2.0:1789760727.373789:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706421281229 time 706478497484 state 0x0 ref 1 00000100:00100000:2.0:1789760727.373808:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508298368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760727.374001:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19808256, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:2.0:1789760727.374012:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54b4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.374034:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760727.374040:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.374048:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54b4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.374066:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54b4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.424731:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972156 is committed 00002000:00010000:3.0:1789760727.430623:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19808256 -> 19812352 00010000:00010000:3.0:1789760727.430656:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54b4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.430671:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54b4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.430680:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54b4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.430691:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760727.430705:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54b4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.430716:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54b4b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.430744:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54b4b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.430758:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54b4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19808256->19812351] (req 19808256->19812351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.430900:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508298368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57091us (57928us total) trans 4294972156 rc 0/0 00000100:00100000:3.0:1789760727.430912:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4868 00000100:00100000:3.0:1789760727.430935:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706421281229 time 706478497484 state 0x2 ref 0 00000100:00100000:2.0:1789760727.438991:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760727.439127:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508298496 00000100:00100000:3.0:1789760727.439140:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508298496 00000100:00100000:3.0:1789760727.439230:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706421281229 time 706478497484 state 0x0 ref 1 00000100:00100000:3.0:1789760727.439250:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706478757900 time 706478497484 state 0x1 ref 1 00000100:00100000:3.0:1789760727.439312:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706478757900 time 706543997742 state 0x0 ref 1 00000100:00100000:3.0:1789760727.439328:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508298496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760727.439392:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19812352, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:3.0:1789760727.439403:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.439423:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760727.439428:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.439435:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.439458:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.485951:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972157 is committed 00002000:00010000:3.0:1789760727.486027:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19812352 -> 19816448 00010000:00010000:3.0:1789760727.486047:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.486064:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.486075:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.486084:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760727.486090:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.486100:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.486140:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.486149:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19812352->19816447] (req 19812352->19816447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.486341:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508298496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46962us (47308us total) trans 4294972157 rc 0/0 00000100:00100000:3.0:1789760727.486353:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4869 00000100:00100000:3.0:1789760727.486366:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706478757900 time 706543997742 state 0x2 ref 0 00000100:00100000:2.0:1789760727.490616:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760727.490685:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508298624 00000100:00100000:3.0:1789760727.490699:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508298624 00000100:00100000:3.0:1789760727.490782:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706478757900 time 706543997742 state 0x0 ref 1 00000100:00100000:3.0:1789760727.490802:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706544258158 time 706543997742 state 0x1 ref 1 00000100:00100000:3.0:1789760727.490840:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706544258158 time 706595549717 state 0x0 ref 1 00000100:00100000:3.0:1789760727.490855:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508298624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760727.490928:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19816448, mtime: 0x6aad94d7, atime: 0x0, ctime: 0x6aad94d7, blocks: 0x9807 00010000:00010000:3.0:1789760727.490939:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.490960:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760727.490965:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.490972:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.490989:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760727.528154:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972158 is committed 00002000:00010000:1.0:1789760727.528316:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19816448 -> 19820544 00002000:00010000:1.0:1789760727.528320:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760727 -> 1789760728 00002000:00010000:1.0:1789760727.528322:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760727 -> 1789760728 00010000:00010000:1.0:1789760727.528338:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.528350:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.528357:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.528362:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760727.528366:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.528373:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.528393:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b59 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.528397:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19816448->19820543] (req 19816448->19820543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.528501:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508298624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37643us (37894us total) trans 4294972158 rc 0/0 00000100:00100000:1.0:1789760727.528513:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4870 00000100:00100000:1.0:1789760727.528526:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706544258158 time 706595549717 state 0x2 ref 0 00000100:00100000:0.0:1789760727.531038:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760727.531200:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508298752 00000100:00100000:3.0:1789760727.531215:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508298752 00000100:00100000:3.0:1789760727.531317:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706544258158 time 706595549717 state 0x0 ref 1 00000100:00100000:3.0:1789760727.531337:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706595810133 time 706595549717 state 0x1 ref 1 00000100:00100000:3.0:1789760727.531362:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706595810133 time 706636072519 state 0x0 ref 1 00000100:00100000:3.0:1789760727.531373:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508298752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760727.531428:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19820544, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:3.0:1789760727.531438:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.531457:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760727.531462:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.531470:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.531488:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760727.561856:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972159 is committed 00002000:00010000:2.0:1789760727.562092:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19820544 -> 19824640 00010000:00010000:2.0:1789760727.562114:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.562129:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.562139:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.562147:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760727.562153:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.562164:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.562197:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.562207:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe54b60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19820544->19824639] (req 19820544->19824639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.562438:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508298752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31059us (31408us total) trans 4294972159 rc 0/0 00000100:00100000:2.0:1789760727.562450:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4871 00000100:00100000:2.0:1789760727.562463:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706595810133 time 706636072519 state 0x2 ref 0 00000100:00100000:2.0:1789760727.565604:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760727.565724:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508298880 00000100:00100000:3.0:1789760727.565739:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508298880 00000100:00100000:3.0:1789760727.565809:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706595810133 time 706636072519 state 0x0 ref 1 00000100:00100000:3.0:1789760727.565887:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706636332935 time 706636072519 state 0x1 ref 1 00000100:00100000:3.0:1789760727.565927:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706636332935 time 706670637169 state 0x0 ref 1 00000100:00100000:3.0:1789760727.565943:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508298880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760727.566009:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19824640, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:3.0:1789760727.566021:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.566043:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760727.566048:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.566057:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.566073:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.601456:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972160 is committed 00002000:00010000:1.0:1789760727.601746:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19824640 -> 19828736 00010000:00010000:1.0:1789760727.601764:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.601774:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.601883:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.601889:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760727.601893:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.601900:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.601920:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b67 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.601925:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe54b67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19824640->19828735] (req 19824640->19828735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.602027:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508298880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36080us (36432us total) trans 4294972160 rc 0/0 00000100:00100000:1.0:1789760727.602037:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4872 00000100:00100000:1.0:1789760727.602049:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706636332935 time 706670637169 state 0x2 ref 0 00000100:00100000:3.0:1789760727.604882:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760727.604988:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508299008 00000100:00100000:1.0:1789760727.604999:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508299008 00000100:00100000:1.0:1789760727.605062:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706636332935 time 706670637169 state 0x0 ref 1 00000100:00100000:1.0:1789760727.605076:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706670897585 time 706670637169 state 0x1 ref 1 00000100:00100000:2.0:1789760727.605692:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706670897585 time 706710397076 state 0x0 ref 1 00000100:00100000:2.0:1789760727.605715:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d47850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508299008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760727.605783:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19828736, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:2.0:1789760727.605794:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b6e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.605813:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760727.605818:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.605824:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.605839:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.640987:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972161 is committed 00002000:00010000:2.0:1789760727.641066:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19828736 -> 19832832 00010000:00010000:2.0:1789760727.641085:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.641100:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.641111:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.641121:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760727.641127:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.641138:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.641164:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.641174:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19828736->19832831] (req 19828736->19832831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.641469:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d47850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508299008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35756us (36594us total) trans 4294972161 rc 0/0 00000100:00100000:2.0:1789760727.641483:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4873 00000100:00100000:2.0:1789760727.641495:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706670897585 time 706710397076 state 0x2 ref 0 00000100:00100000:0.0:1789760727.645509:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760727.645769:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508299136 00000100:00100000:3.0:1789760727.645784:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508299136 00000100:00100000:3.0:1789760727.645859:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706670897585 time 706710397076 state 0x0 ref 1 00000100:00100000:3.0:1789760727.645886:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706710657492 time 706710397076 state 0x1 ref 1 00000100:00100000:3.0:1789760727.645923:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706710657492 time 706750632658 state 0x0 ref 1 00000100:00100000:3.0:1789760727.645941:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508299136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760727.646006:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19832832, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:3.0:1789760727.646017:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.646038:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760727.646043:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.646051:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.646068:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760727.685161:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19832832 -> 19836928 00010000:00010000:3.0:1789760727.685197:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.685222:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.685232:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.685240:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760727.685245:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.685255:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.685279:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.685323:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19832832->19836927] (req 19832832->19836927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.685437:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508299136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39491us (39936us total) trans 4294972162 rc 0/0 00000100:00100000:3.0:1789760727.685449:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4874 00000100:00100000:3.0:1789760727.685461:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706710657492 time 706750632658 state 0x2 ref 0 00000001:00080000:2.0:1789760727.685603:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972162 is committed 00000100:00100000:2.0:1789760727.689785:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760727.689976:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508299264 00000100:00100000:1.0:1789760727.689990:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508299264 00000100:00100000:1.0:1789760727.690068:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706710657492 time 706750632658 state 0x0 ref 1 00000100:00100000:1.0:1789760727.690087:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706750893074 time 706750632658 state 0x1 ref 1 00000100:00100000:1.0:1789760727.690136:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706750893074 time 706794845176 state 0x0 ref 1 00000100:00100000:1.0:1789760727.690150:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508299264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760727.690535:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19836928, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:1.0:1789760727.690548:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54b7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.690568:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760727.690573:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.690581:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54b7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.690599:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54b7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760727.723804:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972163 is committed 00002000:00010000:1.0:1789760727.723905:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19836928 -> 19841024 00010000:00010000:1.0:1789760727.723925:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54b7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.723944:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54b7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.723951:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54b7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.723956:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760727.723960:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54b7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.723966:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54b7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.723987:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54b7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.723992:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54b7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19836928->19841023] (req 19836928->19841023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.724133:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508299264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33978us (34356us total) trans 4294972163 rc 0/0 00000100:00100000:1.0:1789760727.724150:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4875 00000100:00100000:1.0:1789760727.724162:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706750893074 time 706794845176 state 0x2 ref 0 00000100:00100000:0.0:1789760727.728660:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760727.728952:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508299392 00000100:00100000:2.0:1789760727.728967:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508299392 00000100:00100000:2.0:1789760727.729060:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706750893074 time 706794845176 state 0x0 ref 1 00000100:00100000:2.0:1789760727.729082:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706795105592 time 706794845176 state 0x1 ref 1 00000100:00100000:2.0:1789760727.729211:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706795105592 time 706833917727 state 0x0 ref 1 00000100:00100000:2.0:1789760727.729227:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508299392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760727.729380:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19841024, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:2.0:1789760727.729392:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.729415:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760727.729421:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.729429:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.729779:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760727.764557:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19841024 -> 19845120 00010000:00010000:2.0:1789760727.764586:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.764601:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.764611:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.764619:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760727.764626:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.764636:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.764662:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.764671:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19841024->19845119] (req 19841024->19845119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760727.764796:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508299392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35566us (36554us total) trans 4294972164 rc 0/0 00000100:00100000:2.0:1789760727.764809:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4876 00000100:00100000:2.0:1789760727.764822:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706795105592 time 706833917727 state 0x2 ref 0 00000001:00080000:1.0:1789760727.765407:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972164 is committed 00000100:00100000:0.0:1789760727.769871:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760727.770824:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508299520 00000100:00100000:3.0:1789760727.770841:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508299520 00000100:00100000:3.0:1789760727.770981:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706795105592 time 706833917727 state 0x0 ref 1 00000100:00100000:3.0:1789760727.771001:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706834178143 time 706833917727 state 0x1 ref 1 00000100:00100000:3.0:1789760727.771080:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706834178143 time 706875789317 state 0x0 ref 1 00000100:00100000:3.0:1789760727.771096:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508299520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760727.771159:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19845120, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:3.0:1789760727.771171:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54b8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.771202:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760727.771207:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.771215:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54b8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.771232:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54b8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.800008:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972165 is committed 00002000:00010000:3.0:1789760727.800148:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19845120 -> 19849216 00010000:00010000:3.0:1789760727.800169:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54b8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.800195:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54b8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.800205:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54b8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.800212:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760727.800217:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54b8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.800227:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54b8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.800253:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54b8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.800273:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54b8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19845120->19849215] (req 19845120->19849215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.800423:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508299520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29322us (30571us total) trans 4294972165 rc 0/0 00000100:00100000:3.0:1789760727.800434:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4877 00000100:00100000:3.0:1789760727.800446:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706834178143 time 706875789317 state 0x2 ref 0 00010000:00010000:0.0:1789760727.806592:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760727.811993:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760727.812912:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508299648 00000100:00100000:1.0:1789760727.812929:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508299648 00000100:00100000:1.0:1789760727.813010:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706834178143 time 706875789317 state 0x0 ref 1 00000100:00100000:1.0:1789760727.813030:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706876049733 time 706875789317 state 0x1 ref 1 00000100:00100000:1.0:1789760727.813072:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706876049733 time 706917781435 state 0x0 ref 1 00000100:00100000:1.0:1789760727.813086:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508299648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760727.813151:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19849216, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:1.0:1789760727.813163:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54b91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.813191:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:1.0:1789760727.813197:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760727.813204:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54b91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.813220:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54b91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760727.840773:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972166 is committed 00002000:00010000:1.0:1789760727.840950:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19849216 -> 19853312 00010000:00010000:1.0:1789760727.840971:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54b91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.840986:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54b91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.840997:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54b91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.841005:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760727.841011:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54b91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.841022:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54b91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.841047:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54b91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760727.841056:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54b91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19849216->19853311] (req 19849216->19853311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760727.841189:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508299648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28091us (29263us total) trans 4294972166 rc 0/0 00000100:00100000:1.0:1789760727.841206:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4878 00000100:00100000:1.0:1789760727.841219:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706876049733 time 706917781435 state 0x2 ref 0 00000100:00100000:1.0:1789760727.849825:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760727.849966:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508299776 00000100:00100000:2.0:1789760727.849978:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508299776 00000100:00100000:2.0:1789760727.850050:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706876049733 time 706917781435 state 0x0 ref 1 00000100:00100000:2.0:1789760727.850070:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706918041851 time 706917781435 state 0x1 ref 1 00000100:00100000:2.0:1789760727.850096:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706918041851 time 706954806728 state 0x0 ref 1 00000100:00100000:2.0:1789760727.850112:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508299776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760727.850187:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19853312, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:2.0:1789760727.850198:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.850219:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=706s v=(0 0) 00010000:00010000:2.0:1789760727.850225:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760727.850233:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760727.850251:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760727.910371:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972167 is committed 00002000:00010000:3.0:1789760727.910541:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19853312 -> 19857408 00010000:00010000:3.0:1789760727.910562:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.910587:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.910598:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.910607:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760727.910613:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.910623:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.910657:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.910667:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54b98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19853312->19857407] (req 19853312->19857407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760727.910785:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508299776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60670us (60969us total) trans 4294972167 rc 0/0 00000100:00100000:3.0:1789760727.910797:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4879 00000100:00100000:3.0:1789760727.910809:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706918041851 time 706954806728 state 0x2 ref 0 00000100:00100000:0.0:1789760727.916433:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760727.916548:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508299904 00000100:00100000:3.0:1789760727.916563:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508299904 00000100:00100000:3.0:1789760727.916653:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706918041851 time 706954806728 state 0x0 ref 1 00000100:00100000:3.0:1789760727.916672:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706955067144 time 706954806728 state 0x1 ref 1 00000100:00100000:3.0:1789760727.916708:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706955067144 time 707021417548 state 0x0 ref 1 00000100:00100000:3.0:1789760727.916722:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508299904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760727.916791:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19857408, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:3.0:1789760727.916803:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.916823:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760727.916829:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760727.916838:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760727.916854:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760728.018231:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19857408 -> 19861504 00010000:00010000:2.0:1789760728.018261:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.018279:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.018322:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.018331:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760728.018336:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.018356:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.018384:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.018393:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe54b9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19857408->19861503] (req 19857408->19861503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.018512:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508299904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 101786us (102089us total) trans 4294972168 rc 0/0 00000100:00100000:2.0:1789760728.018529:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4880 00000100:00100000:2.0:1789760728.018541:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706955067144 time 707021417548 state 0x2 ref 0 00000001:00080000:1.0:1789760728.018629:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972168 is committed 00000100:00100000:1.0:1789760728.024055:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760728.024142:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508300032 00000100:00100000:0.0:1789760728.024165:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508300032 00000100:00100000:0.0:1789760728.024248:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 706955067144 time 707021417548 state 0x0 ref 1 00000100:00100000:0.0:1789760728.024268:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707021677964 time 707021417548 state 0x1 ref 1 00000100:00100000:0.0:1789760728.024360:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707021677964 time 707129068700 state 0x0 ref 1 00000100:00100000:0.0:1789760728.024375:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508300032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760728.024444:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19861504, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:0.0:1789760728.024456:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ba6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.024476:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760728.024481:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.024489:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ba6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.024507:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ba6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.072036:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972169 is committed 00002000:00010000:0.0:1789760728.072165:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19861504 -> 19865600 00010000:00010000:0.0:1789760728.072192:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ba6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.072209:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ba6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.072219:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ba6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.072227:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760728.072232:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ba6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.072242:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ba6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.072266:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ba6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.072274:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54ba6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19861504->19865599] (req 19861504->19865599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.072469:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508300032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48089us (48419us total) trans 4294972169 rc 0/0 00000100:00100000:0.0:1789760728.072483:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4881 00000100:00100000:0.0:1789760728.072495:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707021677964 time 707129068700 state 0x2 ref 0 00000100:00100000:3.0:1789760728.079017:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760728.079086:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508300160 00000100:00100000:2.0:1789760728.079106:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508300160 00000100:00100000:2.0:1789760728.079189:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707021677964 time 707129068700 state 0x0 ref 1 00000100:00100000:2.0:1789760728.079209:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707129329116 time 707129068700 state 0x1 ref 1 00000100:00100000:2.0:1789760728.079243:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707129329116 time 707183952898 state 0x0 ref 1 00000100:00100000:2.0:1789760728.079258:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508300160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760728.079451:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19865600, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:2.0:1789760728.079463:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54bad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.079482:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760728.079489:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.079496:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54bad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.079512:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54bad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760728.112603:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19865600 -> 19869696 00010000:00010000:3.0:1789760728.112629:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54bad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.112644:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54bad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.112653:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54bad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.112659:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760728.112665:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54bad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.112676:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54bad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.112698:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54bad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.112706:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54bad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19865600->19869695] (req 19865600->19869695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.112816:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508300160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33553us (33806us total) trans 4294972170 rc 0/0 00000100:00100000:3.0:1789760728.112828:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4882 00000100:00100000:3.0:1789760728.112840:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707129329116 time 707183952898 state 0x2 ref 0 00000001:00080000:3.0:1789760728.112889:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972170 is committed 00000100:00100000:2.0:1789760728.116546:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760728.116844:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508300288 00000100:00100000:3.0:1789760728.116858:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508300288 00000100:00100000:3.0:1789760728.116932:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707129329116 time 707183952898 state 0x0 ref 1 00000100:00100000:3.0:1789760728.116952:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707184213314 time 707183952898 state 0x1 ref 1 00000100:00100000:3.0:1789760728.116989:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707184213314 time 707221699059 state 0x0 ref 1 00000100:00100000:3.0:1789760728.117004:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508300288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760728.117070:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19869696, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:3.0:1789760728.117081:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54bb4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.117227:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760728.117233:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.117241:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54bb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.117259:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54bb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.166590:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972171 is committed 00002000:00010000:0.0:1789760728.166760:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19869696 -> 19873792 00010000:00010000:0.0:1789760728.166780:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54bb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.166795:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54bb4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.166806:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54bb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.166814:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760728.166819:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54bb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.166829:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54bb4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.166853:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54bb4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.166861:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe54bb4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19869696->19873791] (req 19869696->19873791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.166985:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508300288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49976us (50451us total) trans 4294972171 rc 0/0 00000100:00100000:0.0:1789760728.166997:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4883 00000100:00100000:0.0:1789760728.167009:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707184213314 time 707221699059 state 0x2 ref 0 00000100:00100000:1.0:1789760728.169577:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760728.169662:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508300416 00000100:00100000:0.0:1789760728.169674:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508300416 00000100:00100000:0.0:1789760728.169740:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707184213314 time 707221699059 state 0x0 ref 1 00000100:00100000:0.0:1789760728.169758:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707221959475 time 707221699059 state 0x1 ref 1 00000100:00100000:0.0:1789760728.169791:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707221959475 time 707274501305 state 0x0 ref 1 00000100:00100000:0.0:1789760728.169803:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508300416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760728.169865:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19873792, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:0.0:1789760728.169875:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bbb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.169895:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760728.169901:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.169909:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.169926:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760728.210545:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19873792 -> 19877888 00010000:00010000:0.0:1789760728.210579:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.210596:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bbb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.210607:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.210616:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760728.210623:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.210633:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bbb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.210660:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bbb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.210669:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bbb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19873792->19877887] (req 19873792->19877887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.210809:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508300416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40998us (41236us total) trans 4294972172 rc 0/0 00000100:00100000:0.0:1789760728.210827:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4884 00000100:00100000:0.0:1789760728.210841:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707221959475 time 707274501305 state 0x2 ref 0 00000001:00080000:3.0:1789760728.211591:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972172 is committed 00000100:00100000:3.0:1789760728.214076:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760728.214256:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508300544 00000100:00100000:2.0:1789760728.214270:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508300544 00000100:00100000:2.0:1789760728.214378:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707221959475 time 707274501305 state 0x0 ref 1 00000100:00100000:2.0:1789760728.214398:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707274761721 time 707274501305 state 0x1 ref 1 00000100:00100000:2.0:1789760728.214423:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707274761721 time 707319133865 state 0x0 ref 1 00000100:00100000:2.0:1789760728.214438:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508300544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760728.214506:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19877888, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:2.0:1789760728.214517:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54bc2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.214537:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760728.214542:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.214550:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54bc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.214567:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54bc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.244931:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972173 is committed 00002000:00010000:3.0:1789760728.245052:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19877888 -> 19881984 00010000:00010000:3.0:1789760728.245073:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54bc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.245157:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54bc2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.245169:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54bc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.245187:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760728.245193:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54bc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.245204:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54bc2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.245229:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54bc2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.245238:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54bc2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19877888->19881983] (req 19877888->19881983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.245478:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508300544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31035us (31410us total) trans 4294972173 rc 0/0 00000100:00100000:3.0:1789760728.245491:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4885 00000100:00100000:3.0:1789760728.245503:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707274761721 time 707319133865 state 0x2 ref 0 00000100:00100000:2.0:1789760728.249645:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760728.249861:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508300672 00000100:00100000:0.0:1789760728.249873:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508300672 00000100:00100000:0.0:1789760728.249938:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707274761721 time 707319133865 state 0x0 ref 1 00000100:00100000:0.0:1789760728.249957:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707319394281 time 707319133865 state 0x1 ref 1 00000100:00100000:0.0:1789760728.250027:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707319394281 time 707354735971 state 0x0 ref 1 00000100:00100000:0.0:1789760728.250042:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508300672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760728.250119:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19881984, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:0.0:1789760728.250129:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bc9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.250148:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760728.250154:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.250162:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.250188:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760728.303232:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19881984 -> 19886080 00010000:00010000:0.0:1789760728.303261:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.303277:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bc9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.303310:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.303319:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760728.303325:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.303335:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bc9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.303361:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bc9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.303369:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54bc9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19881984->19886079] (req 19881984->19886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.303493:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508300672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53447us (53852us total) trans 4294972174 rc 0/0 00000100:00100000:0.0:1789760728.303504:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4886 00000100:00100000:0.0:1789760728.303516:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707319394281 time 707354735971 state 0x2 ref 0 00000001:00080000:0.0:1789760728.303583:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972174 is committed 00000100:00100000:2.0:1789760728.306702:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760728.306796:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508300800 00000100:00100000:0.0:1789760728.306811:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508300800 00000100:00100000:0.0:1789760728.307047:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707319394281 time 707354735971 state 0x0 ref 1 00000100:00100000:0.0:1789760728.307068:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707354996387 time 707354735971 state 0x1 ref 1 00000100:00100000:0.0:1789760728.307154:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707354996387 time 707411863537 state 0x0 ref 1 00000100:00100000:0.0:1789760728.307169:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508300800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760728.307238:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19886080, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:0.0:1789760728.307248:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54bd0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.307267:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760728.307272:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.307279:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54bd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.307328:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54bd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.340992:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972175 is committed 00002000:00010000:2.0:1789760728.341194:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19886080 -> 19890176 00010000:00010000:2.0:1789760728.341216:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54bd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.341230:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54bd0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.341241:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54bd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.341249:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760728.341263:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54bd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.341273:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54bd0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.341486:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54bd0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.341497:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54bd0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19886080->19890175] (req 19886080->19890175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.341677:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508300800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34504us (34984us total) trans 4294972175 rc 0/0 00000100:00100000:2.0:1789760728.341690:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4887 00000100:00100000:2.0:1789760728.341702:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707354996387 time 707411863537 state 0x2 ref 0 00000100:00100000:3.0:1789760728.347426:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760728.347581:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508300928 00000100:00100000:2.0:1789760728.347595:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508300928 00000100:00100000:2.0:1789760728.347671:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707354996387 time 707411863537 state 0x0 ref 1 00000100:00100000:2.0:1789760728.347691:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707412123953 time 707411863537 state 0x1 ref 1 00000100:00100000:2.0:1789760728.347726:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707412123953 time 707452436509 state 0x0 ref 1 00000100:00100000:2.0:1789760728.347741:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508300928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760728.347806:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19890176, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:2.0:1789760728.347817:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54bd7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.347837:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760728.347843:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.347850:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54bd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.347868:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54bd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760728.406942:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19890176 -> 19894272 00010000:00010000:0.0:1789760728.406968:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54bd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.406992:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54bd7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.407003:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54bd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.407012:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760728.407018:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54bd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.407030:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54bd7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.407054:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54bd7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.407063:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54bd7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19890176->19894271] (req 19890176->19894271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.407187:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508300928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59432us (59759us total) trans 4294972176 rc 0/0 00000100:00100000:0.0:1789760728.407200:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4888 00000100:00100000:0.0:1789760728.407212:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707412123953 time 707452436509 state 0x2 ref 0 00000001:00080000:0.0:1789760728.407369:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972176 is committed 00000100:00100000:0.0:1789760728.413352:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760728.413549:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508301056 00000100:00100000:3.0:1789760728.413563:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508301056 00000100:00100000:3.0:1789760728.413655:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707412123953 time 707452436509 state 0x0 ref 1 00000100:00100000:3.0:1789760728.413676:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707452696925 time 707452436509 state 0x1 ref 1 00000100:00100000:3.0:1789760728.413713:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707452696925 time 707518422802 state 0x0 ref 1 00000100:00100000:3.0:1789760728.413729:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508301056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760728.413793:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19894272, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:3.0:1789760728.413804:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54bde lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.413824:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760728.413830:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.413838:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54bde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.413856:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54bde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760728.456778:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19894272 -> 19898368 00010000:00010000:2.0:1789760728.456805:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54bde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.456822:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54bde lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.456831:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54bde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.456838:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760728.456843:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54bde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.456853:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54bde lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.456878:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54bde lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.456886:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe54bde lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19894272->19898367] (req 19894272->19898367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.457019:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508301056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43286us (43675us total) trans 4294972177 rc 0/0 00000100:00100000:2.0:1789760728.457034:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4889 00000100:00100000:2.0:1789760728.457047:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707452696925 time 707518422802 state 0x2 ref 0 00000001:00080000:0.0:1789760728.457111:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972177 is committed 00000100:00100000:2.0:1789760728.465320:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760728.465429:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508301184 00000100:00100000:0.0:1789760728.465444:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508301184 00000100:00100000:0.0:1789760728.465517:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707452696925 time 707518422802 state 0x0 ref 1 00000100:00100000:0.0:1789760728.465538:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707518683218 time 707518422802 state 0x1 ref 1 00000100:00100000:0.0:1789760728.465616:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707518683218 time 707570325408 state 0x0 ref 1 00000100:00100000:0.0:1789760728.465641:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508301184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760728.465706:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19898368, mtime: 0x6aad94d8, atime: 0x0, ctime: 0x6aad94d8, blocks: 0x9807 00010000:00010000:0.0:1789760728.465718:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54be5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.465738:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760728.465743:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.465751:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54be5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.465769:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54be5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.511448:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972178 is committed 00002000:00010000:2.0:1789760728.511643:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19898368 -> 19902464 00002000:00010000:2.0:1789760728.511650:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760728 -> 1789760729 00002000:00010000:2.0:1789760728.511653:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760728 -> 1789760729 00010000:00010000:2.0:1789760728.511670:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54be5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.511684:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54be5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.511693:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54be5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.511699:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760728.511704:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54be5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.511710:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54be5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.511731:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54be5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.511740:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54be5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19898368->19902463] (req 19898368->19902463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.511842:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508301184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46208us (46567us total) trans 4294972178 rc 0/0 00000100:00100000:2.0:1789760728.511855:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4890 00000100:00100000:2.0:1789760728.511867:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707518683218 time 707570325408 state 0x2 ref 0 00000100:00100000:3.0:1789760728.518343:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760728.518743:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508301312 00000100:00100000:0.0:1789760728.518756:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508301312 00000100:00100000:0.0:1789760728.518828:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707518683218 time 707570325408 state 0x0 ref 1 00000100:00100000:0.0:1789760728.518852:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707570585824 time 707570325408 state 0x1 ref 1 00000100:00100000:0.0:1789760728.518888:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707570585824 time 707623598336 state 0x0 ref 1 00000100:00100000:0.0:1789760728.518903:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508301312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760728.518965:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19902464, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9807 00010000:00010000:0.0:1789760728.518975:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54bec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.518996:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760728.519001:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.519008:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54bec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.519025:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54bec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.598882:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972179 is committed 00002000:00010000:2.0:1789760728.599074:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19902464 -> 19906560 00010000:00010000:2.0:1789760728.599095:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54bec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.599110:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54bec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.599119:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54bec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.599127:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760728.599133:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54bec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.599224:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54bec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.599407:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54bec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.599420:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54bec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19902464->19906559] (req 19902464->19906559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.599737:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508301312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 80821us (81442us total) trans 4294972179 rc 0/0 00000100:00100000:2.0:1789760728.599750:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4891 00000100:00100000:2.0:1789760728.599763:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707570585824 time 707623598336 state 0x2 ref 0 00000100:00100000:0.0:1789760728.603860:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760728.604484:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508301440 00000100:00100000:2.0:1789760728.604498:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508301440 00000100:00100000:2.0:1789760728.604589:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707570585824 time 707623598336 state 0x0 ref 1 00000100:00100000:2.0:1789760728.604610:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707623858752 time 707623598336 state 0x1 ref 1 00000100:00100000:0.0:1789760728.604912:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707623858752 time 707709618690 state 0x0 ref 1 00000100:00100000:0.0:1789760728.604931:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508301440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760728.604992:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19906560, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9807 00010000:00010000:0.0:1789760728.605004:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54bf3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.605029:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:0.0:1789760728.605034:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.605042:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54bf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.605059:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54bf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760728.664140:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972180 is committed 00002000:00010000:0.0:1789760728.664215:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19906560 -> 19910656 00010000:00010000:0.0:1789760728.664234:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54bf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.664251:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54bf3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.664260:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54bf3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.664269:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760728.664275:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54bf3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.664304:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54bf3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.664332:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54bf3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.664342:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54bf3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19906560->19910655] (req 19906560->19910655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.664463:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508301440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59530us (60611us total) trans 4294972180 rc 0/0 00000100:00100000:0.0:1789760728.664475:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4892 00000100:00100000:0.0:1789760728.664489:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707623858752 time 707709618690 state 0x2 ref 0 00000100:00100000:1.0:1789760728.669510:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760728.669684:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508301568 00000100:00100000:2.0:1789760728.669703:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508301568 00000100:00100000:2.0:1789760728.669786:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707623858752 time 707709618690 state 0x0 ref 1 00000100:00100000:2.0:1789760728.669805:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707709879106 time 707709618690 state 0x1 ref 1 00000100:00100000:2.0:1789760728.669836:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707709879106 time 707774547372 state 0x0 ref 1 00000100:00100000:2.0:1789760728.669849:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508301568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760728.669908:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19910656, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9807 00010000:00010000:2.0:1789760728.669920:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe54bfa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.669941:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760728.669946:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.669954:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe54bfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.669971:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe54bfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.725046:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972181 is committed 00002000:00010000:3.0:1789760728.725218:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19910656 -> 19914752 00010000:00010000:3.0:1789760728.725234:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe54bfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.725246:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe54bfa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.725255:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe54bfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.725262:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760728.725267:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe54bfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.725275:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe54bfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.725322:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe54bfa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.725331:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe54bfa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19910656->19914751] (req 19910656->19914751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.725411:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508301568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55559us (55904us total) trans 4294972181 rc 0/0 00000100:00100000:3.0:1789760728.725421:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4893 00000100:00100000:3.0:1789760728.725431:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707709879106 time 707774547372 state 0x2 ref 0 00000100:00100000:3.0:1789760728.729508:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760728.730226:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508301696 00000100:00100000:2.0:1789760728.730237:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508301696 00000100:00100000:2.0:1789760728.730314:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707709879106 time 707774547372 state 0x0 ref 1 00000100:00100000:2.0:1789760728.730330:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707774807788 time 707774547372 state 0x1 ref 1 00000100:00100000:2.0:1789760728.730354:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707774807788 time 707835064919 state 0x0 ref 1 00000100:00100000:2.0:1789760728.730434:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508301696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760728.730484:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19914752, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9807 00010000:00010000:2.0:1789760728.730492:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54c01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.730507:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:2.0:1789760728.730512:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760728.730518:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54c01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.730531:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54c01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.771164:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972182 is committed 00002000:00010000:1.0:1789760728.771252:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19914752 -> 19918848 00010000:00010000:1.0:1789760728.771265:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54c01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.771279:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54c01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.771305:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54c01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.771313:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760728.771318:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54c01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.771326:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54c01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.771353:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54c01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.771361:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe54c01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19914752->19918847] (req 19914752->19918847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.771483:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508301696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41048us (41979us total) trans 4294972182 rc 0/0 00000100:00100000:1.0:1789760728.771493:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4894 00000100:00100000:1.0:1789760728.771502:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707774807788 time 707835064919 state 0x2 ref 0 00000100:00100000:0.0:1789760728.774342:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760728.774473:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508301824 00000100:00100000:1.0:1789760728.774482:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508301824 00000100:00100000:1.0:1789760728.774526:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707774807788 time 707835064919 state 0x0 ref 1 00000100:00100000:1.0:1789760728.774540:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707835325335 time 707835064919 state 0x1 ref 1 00000100:00100000:1.0:1789760728.774571:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707835325335 time 707879281425 state 0x0 ref 1 00000100:00100000:1.0:1789760728.774582:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508301824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760728.774627:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19918848, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9807 00010000:00010000:1.0:1789760728.774640:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.774656:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760728.774661:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.774667:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.774679:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760728.813881:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972183 is committed 00002000:00010000:0.0:1789760728.813974:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19918848 -> 19922944 00010000:00010000:0.0:1789760728.813988:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.814003:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.814012:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.814021:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760728.814026:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.814035:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.814056:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.814065:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19918848->19922943] (req 19918848->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760728.814165:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508301824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39579us (39826us total) trans 4294972183 rc 0/0 00000100:00100000:0.0:1789760728.814188:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4895 00000100:00100000:0.0:1789760728.814198:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707835325335 time 707879281425 state 0x2 ref 0 00000100:00100000:2.0:1789760728.824380:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760728.824487:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508301952 00000100:00100000:3.0:1789760728.824496:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508301952 00000100:00100000:3.0:1789760728.824562:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707835325335 time 707879281425 state 0x0 ref 1 00000100:00100000:3.0:1789760728.824576:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707879541841 time 707879281425 state 0x1 ref 1 00000100:00100000:3.0:1789760728.824708:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707879541841 time 707929319184 state 0x0 ref 1 00000100:00100000:3.0:1789760728.824723:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508301952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760728.824769:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19922944, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0x9807 00010000:00010000:3.0:1789760728.824790:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.824804:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760728.824809:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.824816:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.824830:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.834750:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760728.860896:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19922944 -> 19927040 00002000:00010000:3.0:1789760728.860905:0:8110:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:3.0:1789760728.860918:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.860931:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.860940:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.860948:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760728.860952:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.860962:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.860981:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.860989:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19922944->19927039] (req 19922944->19927039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.861079:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508301952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36353us (36704us total) trans 4294972184 rc 0/0 00000100:00100000:3.0:1789760728.861090:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4896 00000100:00100000:3.0:1789760728.861100:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707879541841 time 707929319184 state 0x2 ref 0 00000001:00080000:3.0:1789760728.861137:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972184 is committed 00000100:00100000:3.0:1789760728.868072:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760728.868154:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508302080 00000100:00100000:1.0:1789760728.868163:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508302080 00000100:00100000:1.0:1789760728.868220:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707879541841 time 707929319184 state 0x0 ref 1 00000100:00100000:1.0:1789760728.868234:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707929579600 time 707929319184 state 0x1 ref 1 00000100:00100000:1.0:1789760728.868259:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707929579600 time 707972969947 state 0x0 ref 1 00000100:00100000:1.0:1789760728.868270:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508302080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760728.868346:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19927040, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:1.0:1789760728.868355:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54c16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.868369:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=707s v=(0 0) 00010000:00010000:1.0:1789760728.868374:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.868380:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54c16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.868394:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54c16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760728.892547:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972185 is committed 00002000:00010000:2.0:1789760728.892651:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19927040 -> 19931136 00010000:00010000:2.0:1789760728.892666:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54c16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.892679:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54c16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.892688:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54c16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.892696:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760728.892700:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54c16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.892710:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54c16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.892729:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54c16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760728.892738:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54c16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19927040->19931135] (req 19927040->19931135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760728.892827:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508302080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24553us (24761us total) trans 4294972185 rc 0/0 00000100:00100000:2.0:1789760728.892837:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4897 00000100:00100000:2.0:1789760728.892847:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707929579600 time 707972969947 state 0x2 ref 0 00000100:00100000:2.0:1789760728.896717:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760728.896792:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508302208 00000100:00100000:0.0:1789760728.896802:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508302208 00000100:00100000:0.0:1789760728.896855:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707929579600 time 707972969947 state 0x0 ref 1 00000100:00100000:0.0:1789760728.896870:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707973230363 time 707972969947 state 0x1 ref 1 00000100:00100000:0.0:1789760728.896894:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707973230363 time 708001604749 state 0x0 ref 1 00000100:00100000:0.0:1789760728.896908:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508302208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760728.896954:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19931136, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:0.0:1789760728.896963:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54c1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.896983:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760728.896987:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760728.896993:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54c1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760728.897009:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54c1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760728.926799:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972186 is committed 00002000:00010000:1.0:1789760728.926893:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19931136 -> 19935232 00010000:00010000:1.0:1789760728.926908:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54c1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.926920:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54c1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.926929:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54c1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.926937:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760728.926942:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54c1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.926953:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54c1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.926972:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54c1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.926980:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54c1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19931136->19935231] (req 19931136->19935231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760728.927073:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508302208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30163us (30362us total) trans 4294972186 rc 0/0 00000100:00100000:1.0:1789760728.927812:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4898 00000100:00100000:1.0:1789760728.927823:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707973230363 time 708001604749 state 0x2 ref 0 00000100:00100000:0.0:1789760728.930703:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760728.930786:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508302336 00000100:00100000:3.0:1789760728.930795:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508302336 00000100:00100000:3.0:1789760728.930849:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 707973230363 time 708001604749 state 0x0 ref 1 00000100:00100000:3.0:1789760728.930865:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708001865165 time 708001604749 state 0x1 ref 1 00000100:00100000:3.0:1789760728.930937:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708001865165 time 708035646992 state 0x0 ref 1 00000100:00100000:3.0:1789760728.930949:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508302336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760728.930994:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19935232, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:3.0:1789760728.931003:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.931017:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760728.931022:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760728.931028:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.931043:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.958452:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049891ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534064000:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789760728.958535:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760728.958716:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534064000 00000100:00100000:2.0:1789760728.958724:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534064000 00000100:00100000:2.0:1789760728.958811:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 421 00000100:00100000:2.0:1789760728.958821:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534064000:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760728.958858:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000001:00080000:0.0:1789760728.958865:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972187 is committed 00000100:00100000:2.0:1789760728.958905:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534064000:12345-0@lo:41:osp-pre-1-0.0 Request processed in 85us (370us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760728.958912:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 421 00002000:00010000:3.0:1789760728.958984:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19935232 -> 19939328 00010000:00010000:3.0:1789760728.958997:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.959009:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.959018:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.959025:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760728.959030:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.959038:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.959056:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760728.959064:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19935232->19939327] (req 19935232->19939327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760728.959214:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508302336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28265us (28519us total) trans 4294972187 rc 0/0 00000100:00100000:3.0:1789760728.959224:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4899 00000100:00100000:3.0:1789760728.959232:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708001865165 time 708035646992 state 0x2 ref 0 00000100:00100000:3.0:1789760728.959279:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049891ff100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534064000:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760728.966484:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760728.966627:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508302464 00000100:00100000:1.0:1789760728.966638:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508302464 00000100:00100000:1.0:1789760728.966691:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708001865165 time 708035646992 state 0x0 ref 1 00000100:00100000:1.0:1789760728.966706:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708035907408 time 708035646992 state 0x1 ref 1 00000100:00100000:1.0:1789760728.966735:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708035907408 time 708071445758 state 0x0 ref 1 00000100:00100000:1.0:1789760728.966747:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508302464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760728.966790:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19939328, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:1.0:1789760728.966798:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.966813:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760728.966817:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760728.966823:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760728.966836:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760729.001346:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19939328 -> 19943424 00010000:00010000:1.0:1789760729.001365:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.001379:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.001387:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.001394:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760729.001398:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.001407:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.001426:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.001434:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54c2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19939328->19943423] (req 19939328->19943423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.001539:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508302464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34788us (35059us total) trans 4294972188 rc 0/0 00000100:00100000:1.0:1789760729.001550:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4900 00000100:00100000:1.0:1789760729.001559:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708035907408 time 708071445758 state 0x2 ref 0 00000001:00080000:0.0:1789760729.002178:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972188 is committed 00000100:00100000:1.0:1789760729.005182:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760729.005321:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508302592 00000100:00100000:0.0:1789760729.005338:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508302592 00000100:00080000:1.0:1789760729.005392:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760729.005417:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708035907408 time 708071445758 state 0x0 ref 1 00000100:00100000:0.0:1789760729.005432:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708071706174 time 708071445758 state 0x1 ref 1 00000100:00100000:0.0:1789760729.005471:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708071706174 time 708110172743 state 0x0 ref 1 00000100:00100000:0.0:1789760729.005484:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508302592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760729.005534:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19943424, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:0.0:1789760729.005542:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.005558:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760729.005563:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.005569:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.005582:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760729.030448:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19943424 -> 19947520 00010000:00010000:0.0:1789760729.030466:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.030479:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.030489:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.030497:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760729.030501:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.030511:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.030537:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.030550:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19943424->19947519] (req 19943424->19947519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.030636:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508302592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25150us (25470us total) trans 4294972189 rc 0/0 00000100:00100000:0.0:1789760729.030645:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4901 00000100:00100000:0.0:1789760729.030654:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708071706174 time 708110172743 state 0x2 ref 0 00000001:00080000:2.0:1789760729.031767:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972189 is committed 00000100:00100000:2.0:1789760729.035624:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760729.035770:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508302720 00000100:00100000:0.0:1789760729.035780:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508302720 00000100:00100000:0.0:1789760729.035828:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708071706174 time 708110172743 state 0x0 ref 1 00000100:00100000:0.0:1789760729.035842:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708110433159 time 708110172743 state 0x1 ref 1 00000100:00100000:0.0:1789760729.035860:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708110433159 time 708140571401 state 0x0 ref 1 00000100:00100000:0.0:1789760729.035869:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508302720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760729.035912:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19947520, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:0.0:1789760729.035919:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.035935:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760729.035939:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.035944:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.035962:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760729.072945:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19947520 -> 19951616 00010000:00010000:0.0:1789760729.072963:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.072975:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.072984:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.072990:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760729.072995:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.073004:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.073023:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.073031:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19947520->19951615] (req 19947520->19951615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.073253:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508302720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37377us (37631us total) trans 4294972190 rc 0/0 00000100:00100000:0.0:1789760729.073264:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4902 00000100:00100000:0.0:1789760729.073274:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708110433159 time 708140571401 state 0x2 ref 0 00000001:00080000:2.0:1789760729.074031:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972190 is committed 00000100:00100000:3.0:1789760729.076819:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760729.077010:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508302848 00000100:00100000:1.0:1789760729.077020:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508302848 00000100:00100000:1.0:1789760729.077073:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708110433159 time 708140571401 state 0x0 ref 1 00000100:00100000:1.0:1789760729.077157:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708140831817 time 708140571401 state 0x1 ref 1 00000100:00100000:1.0:1789760729.077216:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708140831817 time 708181926481 state 0x0 ref 1 00000100:00100000:1.0:1789760729.077230:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508302848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760729.077276:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19951616, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:1.0:1789760729.077325:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54c40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.077341:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.077345:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.077352:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54c40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.077366:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54c40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.148360:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972191 is committed 00002000:00010000:0.0:1789760729.148850:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19951616 -> 19955712 00010000:00010000:0.0:1789760729.148866:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54c40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.148878:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54c40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.148887:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54c40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.148894:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760729.148899:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54c40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.148909:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54c40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.148931:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54c40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.148943:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54c40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19951616->19955711] (req 19951616->19955711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.149048:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508302848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 71816us (72233us total) trans 4294972191 rc 0/0 00000100:00100000:0.0:1789760729.149059:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4903 00000100:00100000:0.0:1789760729.149070:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708140831817 time 708181926481 state 0x2 ref 0 00000100:00100000:1.0:1789760729.159940:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760729.160222:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508302976 00000100:00100000:0.0:1789760729.160231:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508302976 00000100:00100000:0.0:1789760729.160314:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708140831817 time 708181926481 state 0x0 ref 1 00000100:00100000:0.0:1789760729.160331:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708182186897 time 708181926481 state 0x1 ref 1 00000100:00100000:0.0:1789760729.160362:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708182186897 time 708265072196 state 0x0 ref 1 00000100:00100000:0.0:1789760729.160376:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508302976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760729.160423:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19955712, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:0.0:1789760729.160431:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.160446:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760729.160451:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.160457:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.160477:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.202278:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972192 is committed 00002000:00010000:2.0:1789760729.202330:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19955712 -> 19959808 00010000:00010000:2.0:1789760729.202346:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.202360:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.202370:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.202379:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760729.202384:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.202393:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.202413:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.202422:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54c47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19955712->19959807] (req 19955712->19959807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.202528:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508302976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42140us (42584us total) trans 4294972192 rc 0/0 00000100:00100000:2.0:1789760729.202539:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4904 00000100:00100000:2.0:1789760729.202550:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708182186897 time 708265072196 state 0x2 ref 0 00000100:00100000:2.0:1789760729.206116:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760729.206218:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508303104 00000100:00100000:0.0:1789760729.206227:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508303104 00000100:00100000:0.0:1789760729.206280:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708182186897 time 708265072196 state 0x0 ref 1 00000100:00100000:0.0:1789760729.206336:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708265332612 time 708265072196 state 0x1 ref 1 00000100:00100000:0.0:1789760729.206420:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708265332612 time 708311128828 state 0x0 ref 1 00000100:00100000:0.0:1789760729.206434:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508303104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760729.206481:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19959808, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:0.0:1789760729.206490:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.206503:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760729.206508:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.206696:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.206714:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.252795:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972193 is committed 00002000:00010000:1.0:1789760729.253049:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19959808 -> 19963904 00010000:00010000:1.0:1789760729.253066:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.253085:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.253093:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.253100:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760729.253104:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.253113:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.253133:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.253141:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54c4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19959808->19963903] (req 19959808->19963903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.253250:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508303104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46812us (47138us total) trans 4294972193 rc 0/0 00000100:00100000:1.0:1789760729.253261:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4905 00000100:00100000:1.0:1789760729.253271:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708265332612 time 708311128828 state 0x2 ref 0 00000100:00100000:3.0:1789760729.263928:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760729.264044:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508303232 00000100:00100000:0.0:1789760729.264054:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508303232 00000100:00100000:0.0:1789760729.264110:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708265332612 time 708311128828 state 0x0 ref 1 00000100:00100000:0.0:1789760729.264126:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708311389244 time 708311128828 state 0x1 ref 1 00000100:00100000:0.0:1789760729.264158:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708311389244 time 708368867932 state 0x0 ref 1 00000100:00100000:0.0:1789760729.264181:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508303232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760729.264227:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19963904, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:0.0:1789760729.264238:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54c55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.264253:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760729.264258:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.264264:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54c55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.264279:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54c55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760729.289857:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19963904 -> 19968000 00010000:00010000:2.0:1789760729.289877:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54c55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.289891:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54c55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.289900:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54c55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.289908:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760729.289912:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54c55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.289921:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54c55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.289940:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54c55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.289948:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54c55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19963904->19967999] (req 19963904->19967999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.290043:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508303232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25860us (26120us total) trans 4294972194 rc 0/0 00000100:00100000:2.0:1789760729.290055:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4906 00000100:00100000:2.0:1789760729.290066:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708311389244 time 708368867932 state 0x2 ref 0 00000001:00080000:1.0:1789760729.290139:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972194 is committed 00000100:00100000:2.0:1789760729.300781:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760729.300897:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508303360 00000100:00100000:0.0:1789760729.300908:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508303360 00000100:00100000:0.0:1789760729.300963:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708311389244 time 708368867932 state 0x0 ref 1 00000100:00100000:0.0:1789760729.300977:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708369128348 time 708368867932 state 0x1 ref 1 00000100:00100000:0.0:1789760729.301054:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708369128348 time 708405764329 state 0x0 ref 1 00000100:00100000:0.0:1789760729.301073:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508303360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760729.301125:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19968000, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:0.0:1789760729.301134:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.301148:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760729.301153:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.301159:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.301183:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.346765:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972195 is committed 00002000:00010000:0.0:1789760729.347074:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19968000 -> 19972096 00010000:00010000:0.0:1789760729.347090:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.347101:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.347109:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.347116:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760729.347120:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.347130:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.347150:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.347159:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19968000->19972095] (req 19968000->19972095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.347265:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508303360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46192us (46490us total) trans 4294972195 rc 0/0 00000100:00100000:0.0:1789760729.347276:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4907 00000100:00100000:0.0:1789760729.347330:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708369128348 time 708405764329 state 0x2 ref 0 00000100:00100000:0.0:1789760729.356926:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760729.357051:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508303488 00000100:00100000:1.0:1789760729.357060:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508303488 00000100:00100000:1.0:1789760729.357122:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708369128348 time 708405764329 state 0x0 ref 1 00000100:00100000:1.0:1789760729.357137:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708406024745 time 708405764329 state 0x1 ref 1 00000100:00100000:1.0:1789760729.357172:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708406024745 time 708461874036 state 0x0 ref 1 00000100:00100000:1.0:1789760729.357185:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508303488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760729.357230:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19972096, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:1.0:1789760729.357239:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54c63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.357255:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.357259:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.357266:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54c63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.357279:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54c63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760729.434975:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972196 is committed 00002000:00010000:1.0:1789760729.435112:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19972096 -> 19976192 00010000:00010000:1.0:1789760729.435127:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54c63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.435139:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54c63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.435153:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54c63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.435161:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760729.435176:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54c63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.435186:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54c63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.435204:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54c63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.435212:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54c63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19972096->19976191] (req 19972096->19976191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.435330:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508303488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 78141us (78407us total) trans 4294972196 rc 0/0 00000100:00100000:1.0:1789760729.435341:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4908 00000100:00100000:1.0:1789760729.435351:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708406024745 time 708461874036 state 0x2 ref 0 00000100:00100000:3.0:1789760729.440380:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760729.440472:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508303616 00000100:00100000:2.0:1789760729.440482:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508303616 00000100:00100000:2.0:1789760729.440546:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708406024745 time 708461874036 state 0x0 ref 1 00000100:00100000:2.0:1789760729.440562:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708462134452 time 708461874036 state 0x1 ref 1 00000100:00100000:2.0:1789760729.440593:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708462134452 time 708545302894 state 0x0 ref 1 00000100:00100000:2.0:1789760729.440607:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508303616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760729.440657:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19976192, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:2.0:1789760729.440667:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54c6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.440683:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760729.440688:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.440695:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54c6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.440710:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54c6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.487072:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972197 is committed 00002000:00010000:2.0:1789760729.489678:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19976192 -> 19980288 00010000:00010000:2.0:1789760729.489700:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54c6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.489713:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54c6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.489722:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54c6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.489730:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760729.489735:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54c6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.489744:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54c6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.489763:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54c6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.489771:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe54c6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19976192->19980287] (req 19976192->19980287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.489875:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508303616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49265us (49500us total) trans 4294972197 rc 0/0 00000100:00100000:2.0:1789760729.489886:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4909 00000100:00100000:2.0:1789760729.489896:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708462134452 time 708545302894 state 0x2 ref 0 00000100:00100000:2.0:1789760729.500345:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760729.500423:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508303744 00000100:00100000:0.0:1789760729.500435:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508303744 00000100:00100000:0.0:1789760729.500495:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708462134452 time 708545302894 state 0x0 ref 1 00000100:00100000:0.0:1789760729.500512:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708545563310 time 708545302894 state 0x1 ref 1 00000100:00100000:0.0:1789760729.500607:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708545563310 time 708605316505 state 0x0 ref 1 00000100:00100000:0.0:1789760729.500622:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508303744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760729.500670:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19980288, mtime: 0x6aad94d9, atime: 0x0, ctime: 0x6aad94d9, blocks: 0xa007 00010000:00010000:0.0:1789760729.500680:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54c71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.500694:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760729.500698:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.500705:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54c71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.500720:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54c71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760729.562318:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19980288 -> 19984384 00002000:00010000:0.0:1789760729.562329:0:14638:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760729 -> 1789760730 00002000:00010000:0.0:1789760729.562331:0:14638:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760729 -> 1789760730 00010000:00010000:0.0:1789760729.562345:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54c71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.562357:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54c71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.562367:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54c71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.562375:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760729.562380:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54c71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.562397:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54c71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.562418:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54c71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.562427:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54c71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19980288->19984383] (req 19980288->19984383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.562529:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508303744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61891us (62177us total) trans 4294972198 rc 0/0 00000100:00100000:0.0:1789760729.562541:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4910 00000100:00100000:0.0:1789760729.562551:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708545563310 time 708605316505 state 0x2 ref 0 00000001:00080000:2.0:1789760729.562777:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972198 is committed 00000100:00100000:0.0:1789760729.570134:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760729.570231:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508303872 00000100:00100000:1.0:1789760729.570241:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508303872 00000100:00100000:1.0:1789760729.570332:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708545563310 time 708605316505 state 0x0 ref 1 00000100:00100000:1.0:1789760729.570351:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708605576921 time 708605316505 state 0x1 ref 1 00000100:00100000:1.0:1789760729.570382:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708605576921 time 708675091939 state 0x0 ref 1 00000100:00100000:1.0:1789760729.570395:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508303872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760729.570444:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19984384, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:1.0:1789760729.570453:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54c78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.570469:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.570474:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.570480:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54c78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.570495:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54c78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.623328:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972199 is committed 00002000:00010000:1.0:1789760729.623428:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19984384 -> 19988480 00010000:00010000:1.0:1789760729.623443:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54c78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.623456:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54c78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.623464:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54c78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.623477:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760729.623481:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54c78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.623490:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54c78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.623509:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54c78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.623527:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54c78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19984384->19988479] (req 19984384->19988479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.623614:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508303872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53215us (53485us total) trans 4294972199 rc 0/0 00000100:00100000:1.0:1789760729.623702:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4911 00000100:00100000:1.0:1789760729.623712:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708605576921 time 708675091939 state 0x2 ref 0 00000100:00100000:1.0:1789760729.628655:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760729.628785:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508304000 00000100:00100000:2.0:1789760729.628795:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508304000 00000100:00100000:2.0:1789760729.628854:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708605576921 time 708675091939 state 0x0 ref 1 00000100:00100000:2.0:1789760729.628870:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708675352355 time 708675091939 state 0x1 ref 1 00000100:00100000:2.0:1789760729.628915:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708675352355 time 708733624748 state 0x0 ref 1 00000100:00100000:2.0:1789760729.628928:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508304000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760729.628978:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19988480, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:2.0:1789760729.628987:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54c7f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.629002:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:2.0:1789760729.629008:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760729.629015:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54c7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.629029:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54c7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.684393:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972200 is committed 00002000:00010000:3.0:1789760729.684454:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19988480 -> 19992576 00010000:00010000:3.0:1789760729.684470:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54c7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.684483:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54c7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.684493:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54c7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.684501:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760729.684506:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54c7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.684534:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54c7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.684555:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54c7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.684564:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe54c7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19988480->19992575] (req 19988480->19992575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.684666:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508304000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55734us (56015us total) trans 4294972200 rc 0/0 00000100:00100000:3.0:1789760729.684676:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4912 00000100:00100000:3.0:1789760729.684686:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708675352355 time 708733624748 state 0x2 ref 0 00000100:00100000:3.0:1789760729.688607:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760729.688804:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508304128 00000100:00100000:0.0:1789760729.688814:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508304128 00000100:00100000:0.0:1789760729.688868:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708675352355 time 708733624748 state 0x0 ref 1 00000100:00100000:0.0:1789760729.688882:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708733885164 time 708733624748 state 0x1 ref 1 00000100:00100000:0.0:1789760729.689002:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708733885164 time 708793711201 state 0x0 ref 1 00000100:00100000:0.0:1789760729.689016:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c32ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508304128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760729.689064:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19992576, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:0.0:1789760729.689085:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.689100:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:0.0:1789760729.689105:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.689112:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.689125:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.721719:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972201 is committed 00002000:00010000:0.0:1789760729.722144:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19992576 -> 19996672 00010000:00010000:0.0:1789760729.722168:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.722181:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.722190:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.722199:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760729.722204:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.722213:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.722242:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.722250:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54c86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19992576->19996671] (req 19992576->19996671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760729.722380:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c32ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508304128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33360us (33778us total) trans 4294972201 rc 0/0 00000100:00100000:0.0:1789760729.722390:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4913 00000100:00100000:0.0:1789760729.722400:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708733885164 time 708793711201 state 0x2 ref 0 00000100:00100000:0.0:1789760729.727756:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760729.727843:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508304256 00000100:00100000:1.0:1789760729.727853:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508304256 00000100:00100000:1.0:1789760729.727979:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708733885164 time 708793711201 state 0x0 ref 1 00000100:00100000:1.0:1789760729.727995:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708793971617 time 708793711201 state 0x1 ref 1 00000100:00100000:1.0:1789760729.728025:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708793971617 time 708832734937 state 0x0 ref 1 00000100:00100000:1.0:1789760729.728039:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508304256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760729.728172:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 19996672, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:1.0:1789760729.728183:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.728198:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.728203:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.728210:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.728224:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760729.771799:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 19996672 -> 20000768 00010000:00010000:1.0:1789760729.771818:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.771834:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.771843:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.771852:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760729.771857:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.771867:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.771888:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.771969:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [19996672->20000767] (req 19996672->20000767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.772115:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508304256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44072us (44364us total) trans 4294972202 rc 0/0 00000100:00100000:1.0:1789760729.772126:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4914 00000100:00100000:1.0:1789760729.772136:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708793971617 time 708832734937 state 0x2 ref 0 00000001:00080000:3.0:1789760729.772266:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972202 is committed 00000100:00100000:1.0:1789760729.776892:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760729.776983:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508304384 00000100:00100000:3.0:1789760729.776994:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508304384 00000100:00100000:3.0:1789760729.777049:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708793971617 time 708832734937 state 0x0 ref 1 00000100:00100000:3.0:1789760729.777066:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708832995353 time 708832734937 state 0x1 ref 1 00000100:00100000:3.0:1789760729.777181:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708832995353 time 708881890265 state 0x0 ref 1 00000100:00100000:3.0:1789760729.777195:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508304384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760729.777243:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20000768, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:3.0:1789760729.777252:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.777268:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760729.777272:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.777278:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.777447:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760729.806680:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972203 is committed 00002000:00010000:3.0:1789760729.806759:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20000768 -> 20004864 00010000:00010000:3.0:1789760729.806774:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.806786:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.806795:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.806802:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760729.806806:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.806815:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.806834:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.806842:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54c94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20000768->20004863] (req 20000768->20004863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.806964:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508304384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29766us (30076us total) trans 4294972203 rc 0/0 00000100:00100000:3.0:1789760729.806975:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4915 00000100:00100000:3.0:1789760729.806984:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708832995353 time 708881890265 state 0x2 ref 0 00000100:00100000:0.0:1789760729.811312:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760729.811388:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508304512 00000100:00100000:1.0:1789760729.811397:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508304512 00000100:00100000:1.0:1789760729.811448:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708832995353 time 708881890265 state 0x0 ref 1 00000100:00100000:1.0:1789760729.811471:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708882150681 time 708881890265 state 0x1 ref 1 00000100:00100000:1.0:1789760729.811494:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708882150681 time 708916205684 state 0x0 ref 1 00000100:00100000:1.0:1789760729.811505:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508304512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760729.811650:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20004864, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:1.0:1789760729.811659:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.811673:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.811677:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.811683:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.811696:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760729.854791:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972204 is committed 00002000:00010000:1.0:1789760729.854912:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20004864 -> 20008960 00010000:00010000:1.0:1789760729.854927:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.854939:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.854948:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.854957:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760729.854962:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.854971:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.854996:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.855004:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54c9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20004864->20008959] (req 20004864->20008959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760729.855181:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508304512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43670us (43896us total) trans 4294972204 rc 0/0 00000100:00100000:1.0:1789760729.855191:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4916 00000100:00100000:1.0:1789760729.855200:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708882150681 time 708916205684 state 0x2 ref 0 00010000:00010000:0.0:1789760729.857923:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760729.859091:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760729.859360:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508304640 00000100:00100000:1.0:1789760729.859370:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508304640 00000100:00100000:1.0:1789760729.859427:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708882150681 time 708916205684 state 0x0 ref 1 00000100:00100000:1.0:1789760729.859442:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708916466100 time 708916205684 state 0x1 ref 1 00000100:00100000:1.0:1789760729.859473:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708916466100 time 708964183562 state 0x0 ref 1 00000100:00100000:1.0:1789760729.859486:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508304640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760729.859702:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20008960, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:1.0:1789760729.859711:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54ca2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.859726:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=708s v=(0 0) 00010000:00010000:1.0:1789760729.859731:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760729.859737:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760729.859750:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760729.895882:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20008960 -> 20013056 00010000:00010000:3.0:1789760729.895901:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54ca2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.895913:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54ca2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.895923:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54ca2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.895930:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760729.895935:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54ca2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.895948:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54ca2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.895967:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54ca2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.895974:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54ca2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20008960->20013055] (req 20008960->20013055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760729.896066:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508304640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36577us (36991us total) trans 4294972205 rc 0/0 00000100:00100000:3.0:1789760729.896154:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4917 00000100:00100000:3.0:1789760729.896174:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708916466100 time 708964183562 state 0x2 ref 0 00000001:00080000:0.0:1789760729.896250:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972205 is committed 00000100:00100000:3.0:1789760729.900323:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760729.900355:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508304768 00000100:00100000:3.0:1789760729.900363:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508304768 00000100:00100000:3.0:1789760729.900413:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708916466100 time 708964183562 state 0x0 ref 1 00000100:00100000:3.0:1789760729.900428:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708964443978 time 708964183562 state 0x1 ref 1 00000100:00100000:3.0:1789760729.900715:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708964443978 time 709005423591 state 0x0 ref 1 00000100:00100000:3.0:1789760729.900729:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c32f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508304768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760729.900778:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20013056, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:3.0:1789760729.900788:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54ca9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.900803:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760729.900808:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760729.900814:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54ca9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760729.900830:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54ca9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760729.992003:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972206 is committed 00002000:00010000:2.0:1789760729.992168:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20013056 -> 20017152 00010000:00010000:2.0:1789760729.992184:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54ca9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.992201:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54ca9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.992210:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54ca9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.992217:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760729.992221:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54ca9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.992230:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54ca9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.992250:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54ca9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760729.992258:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe54ca9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20013056->20017151] (req 20013056->20017151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760729.992478:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c32f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508304768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 91745us (92159us total) trans 4294972206 rc 0/0 00000100:00100000:2.0:1789760729.992489:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4918 00000100:00100000:2.0:1789760729.992499:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708964443978 time 709005423591 state 0x2 ref 0 00000100:00100000:3.0:1789760729.995928:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760729.996029:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508304896 00000100:00100000:0.0:1789760729.996039:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508304896 00000100:00100000:0.0:1789760729.996104:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 708964443978 time 709005423591 state 0x0 ref 1 00000100:00100000:0.0:1789760729.996120:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709005684007 time 709005423591 state 0x1 ref 1 00000100:00100000:0.0:1789760729.996151:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709005684007 time 709100861438 state 0x0 ref 1 00000100:00100000:0.0:1789760729.996174:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508304896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760729.996222:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20017152, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:0.0:1789760729.996232:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54cb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.996248:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760729.996253:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760729.996259:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54cb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760729.996273:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54cb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.021987:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972207 is committed 00002000:00010000:3.0:1789760730.022172:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20017152 -> 20021248 00010000:00010000:3.0:1789760730.022189:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54cb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.022201:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54cb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.022211:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54cb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.022220:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760730.022224:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54cb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.022234:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54cb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.022254:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54cb0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.022263:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54cb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20017152->20021247] (req 20017152->20021247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.022426:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508304896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26250us (26503us total) trans 4294972207 rc 0/0 00000100:00100000:3.0:1789760730.022437:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4919 00000100:00100000:3.0:1789760730.022447:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709005684007 time 709100861438 state 0x2 ref 0 00000100:00100000:3.0:1789760730.025444:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760730.025480:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508305024 00000100:00100000:3.0:1789760730.025488:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508305024 00000100:00100000:3.0:1789760730.025539:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709005684007 time 709100861438 state 0x0 ref 1 00000100:00100000:3.0:1789760730.025554:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709101121854 time 709100861438 state 0x1 ref 1 00000100:00100000:3.0:1789760730.025574:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709101121854 time 709130284929 state 0x0 ref 1 00000100:00100000:3.0:1789760730.025584:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508305024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760730.025626:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20021248, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:3.0:1789760730.025634:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54cb7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.025653:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.025658:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.025663:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54cb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.025676:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54cb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760730.094993:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20021248 -> 20025344 00010000:00010000:0.0:1789760730.095013:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54cb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.095026:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54cb7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.095035:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54cb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.095042:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760730.095046:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54cb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.095056:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54cb7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.095149:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54cb7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.095169:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe54cb7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20021248->20025343] (req 20021248->20025343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.095274:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508305024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 69684us (69831us total) trans 4294972208 rc 0/0 00000100:00100000:0.0:1789760730.095315:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4920 00000100:00100000:0.0:1789760730.095327:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709101121854 time 709130284929 state 0x2 ref 0 00000001:00080000:2.0:1789760730.095416:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972208 is committed 00000100:00100000:3.0:1789760730.099004:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760730.099220:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508305152 00000100:00100000:2.0:1789760730.099231:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508305152 00000100:00100000:2.0:1789760730.099325:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709101121854 time 709130284929 state 0x0 ref 1 00000100:00100000:2.0:1789760730.099342:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709130545345 time 709130284929 state 0x1 ref 1 00000100:00100000:2.0:1789760730.099363:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709130545345 time 709204073644 state 0x0 ref 1 00000100:00100000:2.0:1789760730.099376:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508305152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760730.099423:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20025344, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:2.0:1789760730.099432:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54cbe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.099447:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760730.099451:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.099456:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54cbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.099468:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54cbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.137985:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972209 is committed 00002000:00010000:1.0:1789760730.139557:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20025344 -> 20029440 00010000:00010000:1.0:1789760730.139574:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54cbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.139586:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54cbe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.139595:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54cbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.139602:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760730.139606:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54cbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.139615:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54cbe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.139634:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54cbe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.139646:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe54cbe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20025344->20029439] (req 20025344->20029439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.139781:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508305152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40402us (40781us total) trans 4294972209 rc 0/0 00000100:00100000:1.0:1789760730.139792:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4921 00000100:00100000:1.0:1789760730.139801:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709130545345 time 709204073644 state 0x2 ref 0 00000100:00100000:3.0:1789760730.145235:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760730.145343:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508305280 00000100:00100000:0.0:1789760730.145352:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508305280 00000100:00100000:0.0:1789760730.145405:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709130545345 time 709204073644 state 0x0 ref 1 00000100:00100000:0.0:1789760730.145420:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709204334060 time 709204073644 state 0x1 ref 1 00000100:00100000:0.0:1789760730.145450:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709204334060 time 709250160863 state 0x0 ref 1 00000100:00100000:0.0:1789760730.145474:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508305280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760730.145520:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20029440, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:0.0:1789760730.145529:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54cc5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.145543:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760730.145548:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.145554:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54cc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.145567:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54cc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.186323:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760730.186369:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760730.186391:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760730.186410:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760730.186466:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760730.186496:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508305664 00000100:00100000:2.0:1789760730.186506:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508305664 00000100:00100000:2.0:1789760730.186563:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508305920 00000100:00100000:2.0:1789760730.186567:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508305920 00000100:00100000:2.0:1789760730.186583:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508305536 00000100:00100000:2.0:1789760730.186586:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508305536 00000100:00100000:2.0:1789760730.186599:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508305792 00000100:00100000:2.0:1789760730.186602:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508305792 00000100:00100000:2.0:1789760730.186628:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 869 00000100:00100000:2.0:1789760730.186638:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508305664:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760730.186709:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508305664:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 72us (339us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760730.186717:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 869 00000100:00100000:2.0:1789760730.186734:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 870 00000100:00100000:2.0:1789760730.186741:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508305920:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760730.186769:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508305920:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 28us (377us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760730.186776:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 870 00000100:00100000:2.0:1789760730.186787:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 871 00000100:00100000:2.0:1789760730.186794:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2406:x1876699508305536:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760730.186817:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2406:x1876699508305536:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 24us (406us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760730.186823:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 871 00000100:00100000:2.0:1789760730.186836:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 872 00000100:00100000:2.0:1789760730.186843:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988d4fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2406:x1876699508305792:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760730.186867:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988d4fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2406:x1876699508305792:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 25us (401us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760730.186873:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 872 00000100:00080000:2.0:1789760730.186914:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760730.186919:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760730.187394:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508305408 00000100:00100000:3.0:1789760730.187400:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508305408 00000100:00100000:3.0:1789760730.187432:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 467 00000100:00100000:3.0:1789760730.187440:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04989033450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2407:x1876699508305408:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:3.0:1789760730.187473:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04989033450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2407:x1876699508305408:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 34us (1160us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760730.187480:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 467 00000001:00080000:3.0:1789760730.193170:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972210 is committed 00002000:00010000:0.0:1789760730.193271:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20029440 -> 20033536 00010000:00010000:0.0:1789760730.193310:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54cc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.193324:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54cc5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.193343:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54cc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.193352:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760730.193357:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54cc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.193367:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54cc5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.193388:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54cc5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.193404:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54cc5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20029440->20033535] (req 20029440->20033535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.193500:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508305280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48023us (48269us total) trans 4294972210 rc 0/0 00000100:00100000:0.0:1789760730.193520:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4922 00000100:00100000:0.0:1789760730.193530:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709204334060 time 709250160863 state 0x2 ref 0 00000100:00100000:2.0:1789760730.199684:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760730.199822:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508306048 00000100:00100000:0.0:1789760730.199831:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508306048 00000100:00100000:0.0:1789760730.199884:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709204334060 time 709250160863 state 0x0 ref 1 00000100:00100000:0.0:1789760730.199910:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709250421279 time 709250160863 state 0x1 ref 1 00000100:00100000:0.0:1789760730.199939:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709250421279 time 709304649599 state 0x0 ref 1 00000100:00100000:0.0:1789760730.199952:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508306048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760730.200002:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20033536, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:0.0:1789760730.200011:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54ccc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.200025:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760730.200029:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.200036:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54ccc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.200056:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54ccc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760730.245807:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20033536 -> 20037632 00010000:00010000:1.0:1789760730.245826:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54ccc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.245838:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54ccc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.245847:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54ccc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.245855:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760730.245859:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54ccc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.245869:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54ccc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.245904:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54ccc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.245913:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54ccc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20033536->20037631] (req 20033536->20037631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.246010:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508306048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46054us (46331us total) trans 4294972211 rc 0/0 00000100:00100000:1.0:1789760730.246021:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4923 00000100:00100000:1.0:1789760730.246031:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709250421279 time 709304649599 state 0x2 ref 0 00000001:00080000:3.0:1789760730.249792:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972211 is committed 00000100:00100000:2.0:1789760730.249967:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760730.250059:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508306176 00000100:00100000:1.0:1789760730.250134:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508306176 00000100:00100000:1.0:1789760730.250196:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709250421279 time 709304649599 state 0x0 ref 1 00000100:00100000:1.0:1789760730.250211:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709304910015 time 709304649599 state 0x1 ref 1 00000100:00100000:1.0:1789760730.250277:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709304910015 time 709354985184 state 0x0 ref 1 00000100:00100000:1.0:1789760730.250422:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508306176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760730.250473:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20037632, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:1.0:1789760730.250481:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54cd3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.250496:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760730.250502:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.250573:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54cd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.250590:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54cd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760730.282248:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20037632 -> 20041728 00010000:00010000:3.0:1789760730.282266:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54cd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.282279:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54cd3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.282310:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54cd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.282317:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760730.282321:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54cd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.282329:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54cd3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.282348:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54cd3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.282356:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54cd3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20037632->20041727] (req 20037632->20041727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.282445:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508306176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32023us (32478us total) trans 4294972212 rc 0/0 00000100:00100000:3.0:1789760730.282454:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4924 00000100:00100000:3.0:1789760730.282463:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709304910015 time 709354985184 state 0x2 ref 0 00000100:00100000:3.0:1789760730.286276:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760730.286854:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508306304 00000100:00100000:1.0:1789760730.286863:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508306304 00000100:00100000:1.0:1789760730.286926:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709304910015 time 709354985184 state 0x0 ref 1 00000100:00100000:1.0:1789760730.286941:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709355245600 time 709354985184 state 0x1 ref 1 00000100:00100000:1.0:1789760730.286970:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709355245600 time 709391680722 state 0x0 ref 1 00000100:00100000:1.0:1789760730.286983:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508306304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760730.287029:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20041728, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:1.0:1789760730.287038:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54cda lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.287056:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760730.287061:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.287067:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54cda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.287090:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54cda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.287147:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972212 is committed 00002000:00010000:1.0:1789760730.320219:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20041728 -> 20045824 00010000:00010000:1.0:1789760730.320241:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54cda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.320255:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54cda lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.320265:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54cda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.320273:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760730.320278:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54cda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.320305:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54cda lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.320325:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54cda lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.320334:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54cda lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20041728->20045823] (req 20041728->20045823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.320429:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508306304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33443us (34156us total) trans 4294972213 rc 0/0 00000100:00100000:1.0:1789760730.320444:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4925 00000100:00100000:1.0:1789760730.320455:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709355245600 time 709391680722 state 0x2 ref 0 00000001:00080000:2.0:1789760730.320538:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972213 is committed 00000100:00100000:1.0:1789760730.323941:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760730.324089:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508306432 00000100:00100000:3.0:1789760730.324099:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508306432 00000100:00100000:3.0:1789760730.324153:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709355245600 time 709391680722 state 0x0 ref 1 00000100:00100000:3.0:1789760730.324182:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709391941138 time 709391680722 state 0x1 ref 1 00000100:00100000:3.0:1789760730.324214:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709391941138 time 709428924295 state 0x0 ref 1 00000100:00100000:3.0:1789760730.324227:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508306432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760730.324322:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20045824, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:3.0:1789760730.324331:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54ce1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.324347:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.324352:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.324358:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54ce1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.324372:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54ce1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760730.361445:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20045824 -> 20049920 00010000:00010000:3.0:1789760730.361463:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54ce1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.361476:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54ce1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.361483:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54ce1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.361495:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760730.361499:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54ce1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.361517:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54ce1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.361537:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54ce1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.361544:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54ce1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20045824->20049919] (req 20045824->20049919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.361998:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508306432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37765us (38058us total) trans 4294972214 rc 0/0 00000100:00100000:3.0:1789760730.362009:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4926 00000100:00100000:3.0:1789760730.362019:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709391941138 time 709428924295 state 0x2 ref 0 00000001:00080000:2.0:1789760730.362106:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972214 is committed 00000100:00100000:3.0:1789760730.367779:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760730.367882:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508306560 00000100:00100000:1.0:1789760730.367959:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508306560 00000100:00100000:1.0:1789760730.368013:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709391941138 time 709428924295 state 0x0 ref 1 00000100:00100000:1.0:1789760730.368030:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709429184711 time 709428924295 state 0x1 ref 1 00000100:00100000:1.0:1789760730.368063:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709429184711 time 709472773166 state 0x0 ref 1 00000100:00100000:1.0:1789760730.368075:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508306560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760730.368122:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20049920, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:1.0:1789760730.368131:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54ce8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.368146:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760730.368150:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.368166:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54ce8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.368181:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54ce8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.450353:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972215 is committed 00002000:00010000:3.0:1789760730.451531:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20049920 -> 20054016 00010000:00010000:3.0:1789760730.451550:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54ce8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.451574:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54ce8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.451590:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54ce8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.451597:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760730.451602:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54ce8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.451611:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54ce8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.451640:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54ce8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.451648:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54ce8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20049920->20054015] (req 20049920->20054015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.451751:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508306560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 83673us (83982us total) trans 4294972215 rc 0/0 00000100:00100000:3.0:1789760730.451763:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4927 00000100:00100000:3.0:1789760730.451773:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709429184711 time 709472773166 state 0x2 ref 0 00000100:00100000:1.0:1789760730.471405:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760730.471877:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508306688 00000100:00100000:0.0:1789760730.471888:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508306688 00000100:00100000:0.0:1789760730.471947:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709429184711 time 709472773166 state 0x0 ref 1 00000100:00100000:0.0:1789760730.471962:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709473033582 time 709472773166 state 0x1 ref 1 00000100:00100000:0.0:1789760730.471999:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709473033582 time 709576709114 state 0x0 ref 1 00000100:00100000:0.0:1789760730.472011:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd9b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508306688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760730.472070:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20054016, mtime: 0x6aad94da, atime: 0x0, ctime: 0x6aad94da, blocks: 0xa007 00010000:00010000:0.0:1789760730.472081:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54cef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.472096:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760730.472100:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.472106:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54cef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.472120:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54cef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760730.527054:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972216 is committed 00002000:00010000:3.0:1789760730.527927:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20054016 -> 20058112 00002000:00010000:3.0:1789760730.527934:0:14638:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760730 -> 1789760731 00002000:00010000:3.0:1789760730.527936:0:14638:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760730 -> 1789760731 00010000:00010000:3.0:1789760730.527949:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54cef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.527962:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54cef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.527969:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54cef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.527976:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760730.527981:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54cef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.527988:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54cef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.528006:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54cef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.528013:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54cef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20054016->20058111] (req 20054016->20058111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.528228:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd9b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508306688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56212us (56828us total) trans 4294972216 rc 0/0 00000100:00100000:3.0:1789760730.528238:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4928 00000100:00100000:3.0:1789760730.528247:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709473033582 time 709576709114 state 0x2 ref 0 00000100:00100000:1.0:1789760730.537338:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760730.537563:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508306816 00000100:00100000:3.0:1789760730.537574:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508306816 00000100:00100000:3.0:1789760730.537642:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709473033582 time 709576709114 state 0x0 ref 1 00000100:00100000:3.0:1789760730.537658:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709576969530 time 709576709114 state 0x1 ref 1 00000100:00100000:3.0:1789760730.537692:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709576969530 time 709642401764 state 0x0 ref 1 00000100:00100000:3.0:1789760730.537704:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508306816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760730.537754:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20058112, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:3.0:1789760730.537764:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54cf6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.537779:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.537783:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.537790:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54cf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.537806:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54cf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.574583:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972217 is committed 00002000:00010000:0.0:1789760730.574613:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20058112 -> 20062208 00010000:00010000:0.0:1789760730.574628:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54cf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.574642:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54cf6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.574651:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54cf6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.574659:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760730.574664:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54cf6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.574672:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54cf6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.574689:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54cf6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.574694:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe54cf6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20058112->20062207] (req 20058112->20062207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.574771:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508306816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37065us (37481us total) trans 4294972217 rc 0/0 00000100:00100000:0.0:1789760730.574779:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4929 00000100:00100000:0.0:1789760730.574786:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709576969530 time 709642401764 state 0x2 ref 0 00000100:00100000:2.0:1789760730.577855:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760730.577947:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508306944 00000100:00100000:3.0:1789760730.577956:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508306944 00000100:00100000:3.0:1789760730.578007:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709576969530 time 709642401764 state 0x0 ref 1 00000100:00100000:3.0:1789760730.578023:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709642662180 time 709642401764 state 0x1 ref 1 00000100:00100000:3.0:1789760730.578187:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709642662180 time 709682896063 state 0x0 ref 1 00000100:00100000:3.0:1789760730.578205:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d44850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508306944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760730.578252:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20062208, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:3.0:1789760730.578260:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54cfd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.578274:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.578278:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.578283:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54cfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.578333:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54cfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.607953:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972218 is committed 00002000:00010000:3.0:1789760730.608034:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20062208 -> 20066304 00010000:00010000:3.0:1789760730.608048:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54cfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.608060:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54cfd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.608138:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54cfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.608146:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760730.608151:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54cfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.608169:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54cfd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.608187:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54cfd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.608194:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54cfd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20062208->20066303] (req 20062208->20066303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760730.608445:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d44850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508306944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30239us (30594us total) trans 4294972218 rc 0/0 00000100:00100000:3.0:1789760730.608455:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4930 00000100:00100000:3.0:1789760730.608465:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709642662180 time 709682896063 state 0x2 ref 0 00000100:00100000:1.0:1789760730.613758:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760730.614040:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508307072 00000100:00100000:3.0:1789760730.614049:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508307072 00000100:00100000:3.0:1789760730.614433:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709642662180 time 709682896063 state 0x0 ref 1 00000100:00100000:3.0:1789760730.614512:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709683156479 time 709682896063 state 0x1 ref 1 00000100:00100000:3.0:1789760730.614592:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709683156479 time 709719302142 state 0x0 ref 1 00000100:00100000:3.0:1789760730.614607:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508307072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760730.614654:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20066304, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:3.0:1789760730.614664:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54d04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.614680:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.614685:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.614692:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54d04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.614706:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54d04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760730.653914:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972219 is committed 00002000:00010000:1.0:1789760730.654075:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20066304 -> 20070400 00010000:00010000:1.0:1789760730.654090:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54d04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.654101:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54d04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.654109:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54d04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.654116:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760730.654121:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54d04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.654130:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54d04 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.654149:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54d04 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.654166:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe54d04 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20066304->20070399] (req 20066304->20070399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760730.654264:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd9ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508307072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39655us (40511us total) trans 4294972219 rc 0/0 00000100:00100000:1.0:1789760730.654278:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4931 00000100:00100000:1.0:1789760730.654329:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709683156479 time 709719302142 state 0x2 ref 0 00000100:00100000:1.0:1789760730.663677:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760730.664015:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508307200 00000100:00100000:0.0:1789760730.664025:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508307200 00000100:00100000:0.0:1789760730.664079:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709683156479 time 709719302142 state 0x0 ref 1 00000100:00100000:0.0:1789760730.664094:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709719562558 time 709719302142 state 0x1 ref 1 00000100:00100000:0.0:1789760730.664131:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709719562558 time 709768840972 state 0x0 ref 1 00000100:00100000:0.0:1789760730.664144:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508307200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760730.664204:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20070400, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:0.0:1789760730.664213:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.664226:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760730.664230:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.664236:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.664249:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760730.700140:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972220 is committed 00002000:00010000:0.0:1789760730.700427:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20070400 -> 20074496 00010000:00010000:0.0:1789760730.700443:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.700468:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.700477:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.700483:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760730.700488:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.700497:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d0b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.700626:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d0b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.700634:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20070400->20074495] (req 20070400->20074495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760730.700719:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508307200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36573us (37049us total) trans 4294972220 rc 0/0 00000100:00100000:0.0:1789760730.700729:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4932 00000100:00100000:0.0:1789760730.700739:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709719562558 time 709768840972 state 0x2 ref 0 00000100:00100000:2.0:1789760730.703729:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760730.703909:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508307328 00000100:00100000:3.0:1789760730.703919:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508307328 00000100:00100000:3.0:1789760730.703975:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709719562558 time 709768840972 state 0x0 ref 1 00000100:00100000:3.0:1789760730.703991:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709769101388 time 709768840972 state 0x1 ref 1 00000100:00100000:3.0:1789760730.704025:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709769101388 time 709808735069 state 0x0 ref 1 00000100:00100000:3.0:1789760730.704037:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508307328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760730.704084:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20074496, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:3.0:1789760730.704094:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54d12 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.704110:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760730.704114:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760730.704120:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54d12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760730.704134:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54d12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.754748:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972221 is committed 00002000:00010000:2.0:1789760730.754859:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20074496 -> 20078592 00010000:00010000:2.0:1789760730.754872:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54d12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.754884:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54d12 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.754893:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54d12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.754900:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760730.754904:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54d12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.754917:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54d12 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.754936:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54d12 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.754944:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe54d12 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20074496->20078591] (req 20074496->20078591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.755037:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508307328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50997us (51312us total) trans 4294972221 rc 0/0 00000100:00100000:2.0:1789760730.755047:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4933 00000100:00100000:2.0:1789760730.755057:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709769101388 time 709808735069 state 0x2 ref 0 00000100:00100000:3.0:1789760730.761162:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760730.761267:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508307456 00000100:00100000:1.0:1789760730.761276:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508307456 00000100:00100000:1.0:1789760730.761362:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709769101388 time 709808735069 state 0x0 ref 1 00000100:00100000:1.0:1789760730.761378:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709808995485 time 709808735069 state 0x1 ref 1 00000100:00100000:1.0:1789760730.761409:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709808995485 time 709866119553 state 0x0 ref 1 00000100:00100000:1.0:1789760730.761422:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508307456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760730.761620:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20078592, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:1.0:1789760730.761631:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54d19 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.761646:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:1.0:1789760730.761650:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760730.761657:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54d19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760730.761670:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54d19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.817876:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972222 is committed 00002000:00010000:2.0:1789760730.817972:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20078592 -> 20082688 00010000:00010000:2.0:1789760730.817987:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54d19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.817998:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54d19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.818007:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54d19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.818014:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760730.818020:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54d19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.818028:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54d19 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.818047:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54d19 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.818054:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe54d19 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20078592->20082687] (req 20078592->20082687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.818160:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508307456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56725us (57003us total) trans 4294972222 rc 0/0 00000100:00100000:2.0:1789760730.818170:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4934 00000100:00100000:2.0:1789760730.818178:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709808995485 time 709866119553 state 0x2 ref 0 00000100:00100000:1.0:1789760730.826394:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760730.826874:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508307584 00000100:00100000:0.0:1789760730.826884:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508307584 00000100:00100000:0.0:1789760730.826938:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709808995485 time 709866119553 state 0x0 ref 1 00000100:00100000:0.0:1789760730.826954:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709866379969 time 709866119553 state 0x1 ref 1 00000100:00100000:0.0:1789760730.826987:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709866379969 time 709931697830 state 0x0 ref 1 00000100:00100000:0.0:1789760730.827001:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd9b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508307584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760730.827047:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20082688, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:0.0:1789760730.827055:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d20 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.827069:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:0.0:1789760730.827074:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760730.827080:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760730.827095:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.875799:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972223 is committed 00002000:00010000:2.0:1789760730.875984:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20082688 -> 20086784 00010000:00010000:2.0:1789760730.875997:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.876007:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d20 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.876013:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.876022:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760730.876025:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.876031:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.876045:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d20 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.876049:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d20 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20082688->20086783] (req 20082688->20086783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.876125:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd9b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508307584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49119us (49880us total) trans 4294972223 rc 0/0 00000100:00100000:2.0:1789760730.876136:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4935 00000100:00100000:2.0:1789760730.876145:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709866379969 time 709931697830 state 0x2 ref 0 00010000:00010000:0.0:1789760730.879528:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760730.879917:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760730.880117:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508307712 00000100:00100000:2.0:1789760730.880125:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508307712 00000100:00100000:2.0:1789760730.880208:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709866379969 time 709931697830 state 0x0 ref 1 00000100:00100000:2.0:1789760730.880231:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709931958246 time 709931697830 state 0x1 ref 1 00000100:00100000:2.0:1789760730.880254:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709931958246 time 709984964782 state 0x0 ref 1 00000100:00100000:2.0:1789760730.880490:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508307712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760730.880597:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20086784, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:2.0:1789760730.880605:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d27 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.880621:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=709s v=(0 0) 00010000:00010000:2.0:1789760730.880625:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.880631:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.880645:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760730.908951:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972224 is committed 00002000:00010000:2.0:1789760730.909136:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20086784 -> 20090880 00010000:00010000:2.0:1789760730.909259:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.909273:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d27 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.909283:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.909319:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760730.909325:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.909335:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.909356:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d27 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.909364:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d27 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20086784->20090879] (req 20086784->20090879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760730.909456:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508307712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29004us (29541us total) trans 4294972224 rc 0/0 00000100:00100000:2.0:1789760730.909467:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4936 00000100:00100000:2.0:1789760730.909476:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709931958246 time 709984964782 state 0x2 ref 0 00000100:00100000:3.0:1789760730.915202:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760730.915256:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508307840 00000100:00100000:2.0:1789760730.915266:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508307840 00000100:00100000:2.0:1789760730.915350:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709931958246 time 709984964782 state 0x0 ref 1 00000100:00100000:2.0:1789760730.915365:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709985225198 time 709984964782 state 0x1 ref 1 00000100:00100000:2.0:1789760730.915395:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709985225198 time 710020105449 state 0x0 ref 1 00000100:00100000:2.0:1789760730.915408:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508307840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760730.915454:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20090880, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:2.0:1789760730.915463:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54d2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.915478:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760730.915482:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760730.915489:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54d2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760730.915502:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54d2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.007410:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534064128:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760731.007479:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.007607:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534064640:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.007633:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.007649:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3472:1876699534065152:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.007668:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.007681:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c9c6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3472:1876699534065664:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.007697:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.007715:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:3.0:1789760731.007720:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534064256:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760731.007736:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.007754:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534064768:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.007770:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.007782:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534065280:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.007798:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.007810:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c9c7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534065792:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.007824:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.007850:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3474:1876699534064384:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.008029:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.008055:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534064896:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.008613:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.008629:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534065408:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.008647:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.008661:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c9c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534065920:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.008677:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.008695:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:3.0:1789760731.008700:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534064512:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.008717:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.008731:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534065024:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.008746:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.008759:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c9c7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534065536:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.008774:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.008785:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b644a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534066048:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760731.008800:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.008828:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534064128 00000100:00100000:3.0:1789760731.008834:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534064128 00000100:00100000:3.0:1789760731.008872:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534064256 00000100:00100000:3.0:1789760731.008874:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534064256 00000100:00100000:3.0:1789760731.008884:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534066048 00000100:00100000:3.0:1789760731.008886:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534066048 00000100:00100000:3.0:1789760731.008899:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:3.0:1789760731.008906:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534064128:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760731.008927:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3837788160 avail 3835691008 00000100:00100000:3.0:1789760731.008962:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534064128:12345-0@lo:13:osp-pre-0-0.0 Request processed in 56us (1483us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.008969:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00100000:3.0:1789760731.008983:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 361 00000100:00100000:3.0:1789760731.008988:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534064256:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760731.009008:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760731.009028:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534064256:12345-0@lo:13:osp-pre-1-0.0 Request processed in 41us (1291us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.009034:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 361 00000100:00100000:3.0:1789760731.009043:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 362 00000100:00100000:3.0:1789760731.009048:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534066048:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760731.009055:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760731.009171:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534064640 00000100:00100000:0.0:1789760731.009175:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534064640 00000100:00100000:0.0:1789760731.009201:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534064896 00000100:00100000:0.0:1789760731.009203:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534064896 00000100:00100000:0.0:1789760731.009214:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 422 00000100:00100000:0.0:1789760731.009220:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534064640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760731.009341:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 423 00000100:00100000:1.0:1789760731.009350:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534064896:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760731.010376:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534065280 00000100:00100000:2.0:1789760731.010381:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534065280 00000100:00100000:2.0:1789760731.010408:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534065792 00000100:00100000:2.0:1789760731.010410:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534065792 00000100:00100000:2.0:1789760731.010419:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534065408 00000100:00100000:2.0:1789760731.010421:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534065408 00000100:00100000:2.0:1789760731.010428:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534065920 00000100:00100000:2.0:1789760731.010430:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534065920 00000100:00100000:2.0:1789760731.010442:0:8102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:2.0:1789760731.010448:0:8102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534065280:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760731.014552:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534064640:12345-0@lo:400:kworker.0 Request processed in 5319us (6905us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760731.014571:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 422 00000100:00100000:3.0:1789760731.017458:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534066048:12345-0@lo:13:osp-pre-1-1.0 Request processed in 8401us (8648us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.017473:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 362 00000100:00100000:0.0:1789760731.017652:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b644a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534066048:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760731.019233:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:3.0:1789760731.019251:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534065792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.019360:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534065792:12345-0@lo:400:kworker.0 Request processed in 116us (11534us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.019368:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00100000:3.0:1789760731.019392:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:3.0:1789760731.019399:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534065408:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.019440:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534065408:12345-0@lo:400:kworker.0 Request processed in 42us (10791us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.019446:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000100:00100000:3.0:1789760731.019459:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:3.0:1789760731.019466:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534065920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.019495:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534065920:12345-0@lo:400:kworker.0 Request processed in 30us (10817us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.020031:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000100:00100000:3.0:1789760731.020058:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534065152 00000100:00100000:1.0:1789760731.020211:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534064896:12345-0@lo:400:kworker.0 Request processed in 10861us (11597us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760731.020219:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 423 00000100:00100000:1.0:1789760731.020339:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534065664 00000100:00100000:1.0:1789760731.020345:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534065664 00000100:00100000:1.0:1789760731.020375:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534064768 00000100:00100000:1.0:1789760731.020378:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534064768 00000100:00100000:1.0:1789760731.020391:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534064512 00000100:00100000:1.0:1789760731.020393:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534064512 00000100:00100000:1.0:1789760731.020403:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534065024 00000100:00100000:1.0:1789760731.020407:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534065024 00000100:00100000:1.0:1789760731.020418:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534065536 00000100:00100000:1.0:1789760731.020420:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534065536 00000100:00100000:1.0:1789760731.020433:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 873 00000100:00100000:1.0:1789760731.020439:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3472:x1876699534065664:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760731.020476:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3472:x1876699534065664:12345-0@lo:400:kworker.0 Request processed in 36us (12777us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760731.020482:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 873 00000100:00100000:1.0:1789760731.020494:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 874 00000100:00100000:1.0:1789760731.020768:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534064768:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760731.020803:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534064768:12345-0@lo:400:kworker.0 Request processed in 303us (13031us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760731.020809:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 874 00000100:00100000:1.0:1789760731.020825:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 875 00000100:00100000:1.0:1789760731.020831:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534064512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760731.020855:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534064512:12345-0@lo:400:kworker.0 Request processed in 26us (12138us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760731.020861:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 875 00000100:00100000:1.0:1789760731.020871:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 876 00000100:00100000:1.0:1789760731.020877:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534065024:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760731.020903:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534065024:12345-0@lo:400:kworker.0 Request processed in 27us (12156us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760731.020909:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 876 00000100:00100000:1.0:1789760731.020920:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 877 00000100:00100000:1.0:1789760731.020926:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534065536:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760731.020951:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ffb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534065536:12345-0@lo:400:kworker.0 Request processed in 26us (12177us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760731.020956:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 877 00000100:00100000:1.0:1789760731.020993:0:6459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534064384 00000100:00100000:1.0:1789760731.020998:0:6459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534064384 00000100:00100000:1.0:1789760731.021027:0:6459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:1.0:1789760731.021033:0:6459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04989032450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3474:x1876699534064384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760731.021062:0:6459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04989032450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3474:x1876699534064384:12345-0@lo:400:kworker.0 Request processed in 29us (13186us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.021174:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534065152 00000100:00100000:3.0:1789760731.021207:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 878 00000100:00100000:3.0:1789760731.021213:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3472:x1876699534065152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.021249:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3472:x1876699534065152:12345-0@lo:400:kworker.0 Request processed in 36us (13580us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.021255:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 878 00000100:00080000:3.0:1789760731.021282:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760731.021302:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760731.021305:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789760731.021308:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760731.021316:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760731.021329:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0488421b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534066176:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760731.021365:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760731.021395:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534066176 00000100:00100000:3.0:1789760731.021399:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534066176 00000100:00100000:3.0:1789760731.021418:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 363 00000100:00100000:3.0:1789760731.021423:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3473:x1876699534066176:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1789760731.021437:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3837788160 avail 3835691008 00000100:00100000:3.0:1789760731.021465:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3473:x1876699534066176:12345-0@lo:13:osp-pre-0-1.0 Request processed in 42us (100us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760731.021471:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 363 00000100:00100000:1.0:1789760731.021681:0:6459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:3.0:1789760731.022256:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0488421b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534066176:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760731.022328:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534064256:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760731.022342:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534064768:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.022353:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c9c7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534065792:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.022370:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534064128:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760731.022383:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534064640:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.022393:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3472:1876699534065152:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.022405:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c9c6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3472:1876699534065664:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.022443:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534064512:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.022454:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534065024:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.022464:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c9c7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534065536:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.022473:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3474:1876699534064384:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.022482:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534064896:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.022491:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534065408:0@lo:400:kworker.0 00000100:00100000:3.0:1789760731.023596:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c9c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534065920:0@lo:400:kworker.0 00000100:00100000:2.0:1789760731.024675:0:8102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534065280:12345-0@lo:400:kworker.0 Request processed in 14221us (16869us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760731.024687:0:8102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00000100:00100000:3.0:1789760731.024836:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534065280:0@lo:400:kworker.0 00000001:00080000:1.0:1789760731.031213:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972225 is committed 00002000:00010000:0.0:1789760731.031648:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20090880 -> 20094976 00010000:00010000:0.0:1789760731.031664:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54d2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.031676:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54d2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.031684:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54d2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.031690:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760731.031695:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54d2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.031703:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54d2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.031723:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54d2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.031730:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe54d2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20090880->20094975] (req 20090880->20094975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.031823:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508307840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 116413us (116634us total) trans 4294972225 rc 0/0 00000100:00100000:0.0:1789760731.031833:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4937 00000100:00100000:0.0:1789760731.031844:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709985225198 time 710020105449 state 0x2 ref 0 00000100:00100000:1.0:1789760731.036701:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760731.036801:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508307968 00000100:00100000:2.0:1789760731.036810:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508307968 00000100:00100000:2.0:1789760731.036905:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 709985225198 time 710020105449 state 0x0 ref 1 00000100:00100000:2.0:1789760731.036920:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710020365865 time 710020105449 state 0x1 ref 1 00000100:00100000:2.0:1789760731.036980:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710020365865 time 710141689185 state 0x0 ref 1 00000100:00100000:2.0:1789760731.036993:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508307968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760731.037047:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20094976, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:2.0:1789760731.037056:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.037143:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760731.037147:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.037162:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.037177:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760731.037610:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760731.065813:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20094976 -> 20099072 00010000:00010000:2.0:1789760731.065832:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.065845:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.065853:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.065860:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760731.065865:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.065874:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d35 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.065892:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d35 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.065900:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d35 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20094976->20099071] (req 20094976->20099071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.065989:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508307968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28994us (29293us total) trans 4294972226 rc 0/0 00000100:00100000:2.0:1789760731.066001:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4938 00000100:00100000:2.0:1789760731.066010:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710020365865 time 710141689185 state 0x2 ref 0 00000100:00100000:2.0:1789760731.069770:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760731.069895:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508308096 00000100:00100000:3.0:1789760731.069905:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508308096 00000100:00100000:3.0:1789760731.069956:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710020365865 time 710141689185 state 0x0 ref 1 00000100:00100000:3.0:1789760731.069971:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710141949601 time 710141689185 state 0x1 ref 1 00000100:00100000:3.0:1789760731.070002:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710141949601 time 710174712888 state 0x0 ref 1 00000100:00100000:3.0:1789760731.070013:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508308096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760731.070059:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20099072, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:3.0:1789760731.070073:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54d3c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.070089:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760731.070093:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.070099:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54d3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.070111:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54d3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.073580:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972226 is committed 00000001:00080000:3.0:1789760731.091522:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972227 is committed 00002000:00010000:2.0:1789760731.091713:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20099072 -> 20103168 00010000:00010000:2.0:1789760731.091729:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54d3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.091742:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54d3c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.091750:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54d3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.091762:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760731.091767:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54d3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.091775:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54d3c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.091794:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54d3c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.091801:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe54d3c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20099072->20103167] (req 20099072->20103167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.091929:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508308096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 21912us (22164us total) trans 4294972227 rc 0/0 00000100:00100000:2.0:1789760731.091940:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4939 00000100:00100000:2.0:1789760731.091949:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710141949601 time 710174712888 state 0x2 ref 0 00000100:00100000:3.0:1789760731.095375:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760731.095573:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508308224 00000100:00100000:0.0:1789760731.095584:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508308224 00000100:00100000:0.0:1789760731.095641:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710141949601 time 710174712888 state 0x0 ref 1 00000100:00100000:0.0:1789760731.095657:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710174973304 time 710174712888 state 0x1 ref 1 00000100:00100000:0.0:1789760731.095733:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710174973304 time 710200441836 state 0x0 ref 1 00000100:00100000:0.0:1789760731.095747:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508308224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760731.095801:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20103168, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:0.0:1789760731.095811:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.095827:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760731.095832:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.095838:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.095855:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760731.119223:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20103168 -> 20107264 00010000:00010000:2.0:1789760731.119242:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.119255:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.119266:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.119271:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760731.119274:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.119280:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.119309:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d43 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.119315:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20103168->20107263] (req 20103168->20107263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.119538:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508308224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23789us (24169us total) trans 4294972228 rc 0/0 00000100:00100000:2.0:1789760731.119549:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4940 00000100:00100000:2.0:1789760731.119558:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710174973304 time 710200441836 state 0x2 ref 0 00000001:00080000:2.0:1789760731.119597:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972228 is committed 00000100:00100000:2.0:1789760731.125769:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760731.125931:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508308352 00000100:00100000:0.0:1789760731.125942:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508308352 00000100:00100000:0.0:1789760731.126015:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710174973304 time 710200441836 state 0x0 ref 1 00000100:00100000:0.0:1789760731.126031:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710200702252 time 710200441836 state 0x1 ref 1 00000100:00100000:0.0:1789760731.126101:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710200702252 time 710230810809 state 0x0 ref 1 00000100:00100000:0.0:1789760731.126115:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508308352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760731.126174:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20107264, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:0.0:1789760731.126184:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54d4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.126200:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760731.126205:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.126210:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54d4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.126224:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54d4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760731.159939:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20107264 -> 20111360 00010000:00010000:2.0:1789760731.159961:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54d4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.159976:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54d4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.159986:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54d4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.159993:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760731.159997:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54d4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.160006:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54d4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.160027:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54d4a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.160036:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54d4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20107264->20111359] (req 20107264->20111359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.160129:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508308352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34013us (34367us total) trans 4294972229 rc 0/0 00000100:00100000:2.0:1789760731.160140:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4941 00000100:00100000:2.0:1789760731.160160:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710200702252 time 710230810809 state 0x2 ref 0 00000001:00080000:1.0:1789760731.160236:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972229 is committed 00000100:00100000:0.0:1789760731.164409:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760731.164627:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508308480 00000100:00100000:2.0:1789760731.164638:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508308480 00000100:00100000:2.0:1789760731.164696:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710200702252 time 710230810809 state 0x0 ref 1 00000100:00100000:2.0:1789760731.164711:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710231071225 time 710230810809 state 0x1 ref 1 00000100:00100000:2.0:1789760731.164880:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710231071225 time 710269589103 state 0x0 ref 1 00000100:00100000:2.0:1789760731.164895:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508308480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760731.164945:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20111360, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:2.0:1789760731.164954:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.164970:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760731.164974:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.164980:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.164995:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.218866:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972230 is committed 00002000:00010000:2.0:1789760731.218973:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20111360 -> 20115456 00010000:00010000:2.0:1789760731.218989:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.219002:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.219017:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.219024:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760731.219029:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.219038:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.219058:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d51 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.219077:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20111360->20115455] (req 20111360->20115455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.219178:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508308480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54281us (54770us total) trans 4294972230 rc 0/0 00000100:00100000:2.0:1789760731.219189:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4942 00000100:00100000:2.0:1789760731.219198:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710231071225 time 710269589103 state 0x2 ref 0 00000100:00100000:3.0:1789760731.230616:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760731.230727:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508308608 00000100:00100000:2.0:1789760731.230737:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508308608 00000100:00100000:2.0:1789760731.230789:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710231071225 time 710269589103 state 0x0 ref 1 00000100:00100000:2.0:1789760731.230803:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710269849519 time 710269589103 state 0x1 ref 1 00000100:00100000:2.0:1789760731.230833:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710269849519 time 710335543063 state 0x0 ref 1 00000100:00100000:2.0:1789760731.230845:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508308608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760731.230891:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20115456, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:2.0:1789760731.230899:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.230915:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760731.230919:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.230925:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.230939:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760731.269113:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972231 is committed 00002000:00010000:0.0:1789760731.269267:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20115456 -> 20119552 00010000:00010000:0.0:1789760731.269282:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.269327:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.269341:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.269348:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760731.269352:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.269360:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.269380:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d58 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.269388:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54d58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20115456->20119551] (req 20115456->20119551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.269471:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508308608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38621us (38868us total) trans 4294972231 rc 0/0 00000100:00100000:0.0:1789760731.269481:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4943 00000100:00100000:0.0:1789760731.269491:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710269849519 time 710335543063 state 0x2 ref 0 00000100:00100000:0.0:1789760731.274863:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760731.274964:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508308736 00000100:00100000:2.0:1789760731.274973:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508308736 00000100:00100000:2.0:1789760731.275024:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710269849519 time 710335543063 state 0x0 ref 1 00000100:00100000:2.0:1789760731.275038:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710335803479 time 710335543063 state 0x1 ref 1 00000100:00100000:2.0:1789760731.275071:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710335803479 time 710379775702 state 0x0 ref 1 00000100:00100000:2.0:1789760731.275083:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508308736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760731.275127:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20119552, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:2.0:1789760731.275137:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54d5f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.275162:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760731.275166:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.275173:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54d5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.275187:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54d5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.307204:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972232 is committed 00002000:00010000:3.0:1789760731.307421:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20119552 -> 20123648 00010000:00010000:3.0:1789760731.307503:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54d5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.307517:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54d5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.307527:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54d5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.307534:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760731.307539:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54d5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.307549:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54d5f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.307568:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54d5f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.307577:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54d5f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20119552->20123647] (req 20119552->20123647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.307668:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508308736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32582us (32809us total) trans 4294972232 rc 0/0 00000100:00100000:3.0:1789760731.307679:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4944 00000100:00100000:3.0:1789760731.307688:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710335803479 time 710379775702 state 0x2 ref 0 00000100:00100000:1.0:1789760731.314694:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760731.314852:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508308864 00000100:00100000:2.0:1789760731.314862:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508308864 00000100:00100000:2.0:1789760731.314917:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710335803479 time 710379775702 state 0x0 ref 1 00000100:00100000:2.0:1789760731.314932:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710380036118 time 710379775702 state 0x1 ref 1 00000100:00100000:2.0:1789760731.314957:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710380036118 time 710419668203 state 0x0 ref 1 00000100:00100000:2.0:1789760731.314969:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508308864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760731.315018:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20123648, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:2.0:1789760731.315027:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54d66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.315042:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789760731.315047:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760731.315053:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54d66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.315067:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54d66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.347205:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972233 is committed 00002000:00010000:2.0:1789760731.350084:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20123648 -> 20127744 00010000:00010000:2.0:1789760731.350107:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54d66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.350120:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54d66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.350128:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54d66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.350135:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760731.350140:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54d66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.350157:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54d66 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.350179:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54d66 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.350187:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54d66 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20123648->20127743] (req 20123648->20127743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.350372:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508308864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35399us (35681us total) trans 4294972233 rc 0/0 00000100:00100000:2.0:1789760731.350384:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4945 00000100:00100000:2.0:1789760731.350393:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710380036118 time 710419668203 state 0x2 ref 0 00000100:00100000:1.0:1789760731.358329:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760731.358413:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508308992 00000100:00100000:0.0:1789760731.358423:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508308992 00000100:00100000:0.0:1789760731.358486:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710380036118 time 710419668203 state 0x0 ref 1 00000100:00100000:0.0:1789760731.358564:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710419928619 time 710419668203 state 0x1 ref 1 00000100:00100000:0.0:1789760731.358597:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710419928619 time 710463307077 state 0x0 ref 1 00000100:00100000:0.0:1789760731.358610:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508308992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760731.358721:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20127744, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:0.0:1789760731.358730:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.358748:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760731.358753:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.358760:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.358774:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.385564:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972234 is committed 00002000:00010000:0.0:1789760731.385852:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20127744 -> 20131840 00010000:00010000:0.0:1789760731.385867:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.385878:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.385886:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.385893:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760731.385898:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.385906:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.385927:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.385935:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20127744->20131839] (req 20127744->20131839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.386029:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508308992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27416us (27704us total) trans 4294972234 rc 0/0 00000100:00100000:0.0:1789760731.386039:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4946 00000100:00100000:0.0:1789760731.386049:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710419928619 time 710463307077 state 0x2 ref 0 00000100:00100000:0.0:1789760731.391043:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760731.391225:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508309120 00000100:00100000:3.0:1789760731.391235:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508309120 00000100:00100000:3.0:1789760731.391316:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710419928619 time 710463307077 state 0x0 ref 1 00000100:00100000:3.0:1789760731.391335:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710463567493 time 710463307077 state 0x1 ref 1 00000100:00100000:3.0:1789760731.391366:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710463567493 time 710496077000 state 0x0 ref 1 00000100:00100000:3.0:1789760731.391378:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508309120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760731.391423:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20131840, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:3.0:1789760731.391433:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.391448:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760731.391452:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.391458:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.391471:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.419942:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972235 is committed 00002000:00010000:3.0:1789760731.420175:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20131840 -> 20135936 00010000:00010000:3.0:1789760731.420473:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.420487:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.420496:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.420812:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760731.420818:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.420828:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.420848:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d74 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.420856:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d74 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20131840->20135935] (req 20131840->20135935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.421021:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508309120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29639us (29981us total) trans 4294972235 rc 0/0 00000100:00100000:3.0:1789760731.421032:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4947 00000100:00100000:3.0:1789760731.421043:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710463567493 time 710496077000 state 0x2 ref 0 00000100:00100000:2.0:1789760731.425815:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760731.425949:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508309248 00000100:00100000:0.0:1789760731.425960:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508309248 00000100:00100000:0.0:1789760731.426020:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710463567493 time 710496077000 state 0x0 ref 1 00000100:00100000:0.0:1789760731.426036:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710496337416 time 710496077000 state 0x1 ref 1 00000100:00100000:0.0:1789760731.426104:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710496337416 time 710530813587 state 0x0 ref 1 00000100:00100000:0.0:1789760731.426117:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508309248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760731.426177:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20135936, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:0.0:1789760731.426194:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.426209:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760731.426214:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.426220:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.426232:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760731.456895:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972236 is committed 00002000:00010000:1.0:1789760731.456955:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20135936 -> 20140032 00010000:00010000:1.0:1789760731.456970:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.456983:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.456990:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.456995:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760731.456998:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.457004:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.457020:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d7b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.457025:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54d7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20135936->20140031] (req 20135936->20140031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.457370:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508309248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31249us (31559us total) trans 4294972236 rc 0/0 00000100:00100000:1.0:1789760731.457381:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4948 00000100:00100000:1.0:1789760731.457390:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710496337416 time 710530813587 state 0x2 ref 0 00000100:00100000:1.0:1789760731.461022:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760731.461116:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508309376 00000100:00100000:0.0:1789760731.461127:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508309376 00000100:00100000:0.0:1789760731.461191:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710496337416 time 710530813587 state 0x0 ref 1 00000100:00100000:0.0:1789760731.461205:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710531074003 time 710530813587 state 0x1 ref 1 00000100:00100000:0.0:1789760731.461237:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710531074003 time 710565946934 state 0x0 ref 1 00000100:00100000:0.0:1789760731.461249:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508309376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760731.461342:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20140032, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:0.0:1789760731.461352:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d82 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.461374:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760731.461378:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.461384:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.461399:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760731.498665:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20140032 -> 20144128 00010000:00010000:2.0:1789760731.498712:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.498725:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d82 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.498734:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.498741:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760731.498746:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.498755:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.498780:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d82 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.498788:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54d82 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20140032->20144127] (req 20140032->20144127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.498889:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508309376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37636us (37871us total) trans 4294972237 rc 0/0 00000100:00100000:2.0:1789760731.498899:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4949 00000100:00100000:2.0:1789760731.498909:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710531074003 time 710565946934 state 0x2 ref 0 00000001:00080000:1.0:1789760731.498985:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972237 is committed 00000100:00100000:0.0:1789760731.503688:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760731.504072:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508309504 00000100:00100000:3.0:1789760731.504084:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508309504 00000100:00100000:3.0:1789760731.504140:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710531074003 time 710565946934 state 0x0 ref 1 00000100:00100000:3.0:1789760731.504165:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710566207350 time 710565946934 state 0x1 ref 1 00000100:00100000:3.0:1789760731.504416:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710566207350 time 710609124965 state 0x0 ref 1 00000100:00100000:3.0:1789760731.504432:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508309504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760731.504483:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20144128, mtime: 0x6aad94db, atime: 0x0, ctime: 0x6aad94db, blocks: 0xa007 00010000:00010000:3.0:1789760731.504492:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d89 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.504516:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760731.504521:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.504527:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.504542:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.529725:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972238 is committed 00002000:00010000:3.0:1789760731.529840:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20144128 -> 20148224 00002000:00010000:3.0:1789760731.529844:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760731 -> 1789760732 00002000:00010000:3.0:1789760731.529847:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760731 -> 1789760732 00010000:00010000:3.0:1789760731.529859:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.529871:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d89 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.529891:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.529899:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760731.529903:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.529912:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d89 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.529933:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d89 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.529941:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d89 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20144128->20148223] (req 20144128->20148223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.530036:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508309504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25603us (26353us total) trans 4294972238 rc 0/0 00000100:00100000:3.0:1789760731.530045:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4950 00000100:00100000:3.0:1789760731.530054:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710566207350 time 710609124965 state 0x2 ref 0 00000100:00100000:0.0:1789760731.537788:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760731.538413:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508309632 00000100:00100000:1.0:1789760731.538423:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508309632 00000100:00100000:1.0:1789760731.538482:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710566207350 time 710609124965 state 0x0 ref 1 00000100:00100000:1.0:1789760731.538497:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710609385381 time 710609124965 state 0x1 ref 1 00000100:00100000:1.0:1789760731.538576:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710609385381 time 710643285943 state 0x0 ref 1 00000100:00100000:1.0:1789760731.538589:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508309632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760731.538636:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20148224, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:1.0:1789760731.538646:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54d90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.538662:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760731.538667:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.538674:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54d90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.538694:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54d90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.569744:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972239 is committed 00002000:00010000:1.0:1789760731.569859:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20148224 -> 20152320 00010000:00010000:1.0:1789760731.569884:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54d90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.569898:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54d90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.569906:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54d90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.569913:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760731.569918:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54d90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.569927:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54d90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.569947:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54d90 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.569955:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54d90 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20148224->20152319] (req 20148224->20152319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.570052:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508309632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31460us (32268us total) trans 4294972239 rc 0/0 00000100:00100000:1.0:1789760731.570062:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4951 00000100:00100000:1.0:1789760731.570073:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710609385381 time 710643285943 state 0x2 ref 0 00000100:00100000:2.0:1789760731.575406:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760731.575610:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508309760 00000100:00100000:0.0:1789760731.575620:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508309760 00000100:00100000:0.0:1789760731.575678:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710609385381 time 710643285943 state 0x0 ref 1 00000100:00100000:0.0:1789760731.575695:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710643546359 time 710643285943 state 0x1 ref 1 00000100:00100000:0.0:1789760731.575768:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710643546359 time 710680477619 state 0x0 ref 1 00000100:00100000:0.0:1789760731.575783:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508309760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760731.575832:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20152320, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:0.0:1789760731.575841:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d97 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.575858:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760731.575863:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.575869:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.575896:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760731.607570:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20152320 -> 20156416 00000001:00080000:2.0:1789760731.607570:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972240 is committed 00010000:00010000:0.0:1789760731.607586:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.607600:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.607609:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.607616:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760731.607621:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.607630:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.607650:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d97 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.607663:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54d97 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20152320->20156415] (req 20152320->20156415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.607755:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508309760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31969us (32354us total) trans 4294972240 rc 0/0 00000100:00100000:0.0:1789760731.607765:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4952 00000100:00100000:0.0:1789760731.607775:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710643546359 time 710680477619 state 0x2 ref 0 00000100:00100000:0.0:1789760731.615923:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760731.616382:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508309888 00000100:00100000:3.0:1789760731.616393:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508309888 00000100:00100000:3.0:1789760731.616449:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710643546359 time 710680477619 state 0x0 ref 1 00000100:00100000:3.0:1789760731.616465:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710680738035 time 710680477619 state 0x1 ref 1 00000100:00100000:3.0:1789760731.616496:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710680738035 time 710721206676 state 0x0 ref 1 00000100:00100000:3.0:1789760731.616509:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508309888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760731.616556:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20156416, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:3.0:1789760731.616566:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d9e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.616582:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760731.616586:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.616593:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.616607:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760731.649251:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20156416 -> 20160512 00010000:00010000:0.0:1789760731.649272:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.649304:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d9e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.649313:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.649320:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760731.649325:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.649333:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d9e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.649356:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d9e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.649363:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe54d9e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20156416->20160511] (req 20156416->20160511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760731.649462:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508309888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32949us (33542us total) trans 4294972241 rc 0/0 00000100:00100000:0.0:1789760731.649473:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4953 00000100:00100000:0.0:1789760731.649482:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710680738035 time 710721206676 state 0x2 ref 0 00000001:00080000:1.0:1789760731.650116:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972241 is committed 00000100:00100000:2.0:1789760731.663558:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760731.663954:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508310016 00000100:00100000:1.0:1789760731.663963:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508310016 00000100:00100000:1.0:1789760731.664014:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710680738035 time 710721206676 state 0x0 ref 1 00000100:00100000:1.0:1789760731.664029:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710721467092 time 710721206676 state 0x1 ref 1 00000100:00100000:1.0:1789760731.664252:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710721467092 time 710768960403 state 0x0 ref 1 00000100:00100000:1.0:1789760731.664266:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508310016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760731.664647:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20160512, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:1.0:1789760731.664658:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54da5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.664674:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:1.0:1789760731.664678:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760731.664684:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54da5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.664698:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54da5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.705691:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972242 is committed 00002000:00010000:3.0:1789760731.705790:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20160512 -> 20164608 00010000:00010000:3.0:1789760731.705818:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54da5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.705830:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54da5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.705837:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54da5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.705844:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760731.705848:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54da5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.705855:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54da5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.705942:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54da5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.705951:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54da5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20160512->20164607] (req 20160512->20164607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760731.706082:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508310016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41813us (42589us total) trans 4294972242 rc 0/0 00000100:00100000:3.0:1789760731.706094:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4954 00000100:00100000:3.0:1789760731.706103:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710721467092 time 710768960403 state 0x2 ref 0 00000100:00100000:3.0:1789760731.736435:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760731.736570:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508310144 00000100:00100000:0.0:1789760731.736580:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508310144 00000100:00100000:0.0:1789760731.736632:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710721467092 time 710768960403 state 0x0 ref 1 00000100:00100000:0.0:1789760731.736646:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710769220819 time 710768960403 state 0x1 ref 1 00000100:00100000:0.0:1789760731.736675:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710769220819 time 710841385504 state 0x0 ref 1 00000100:00100000:0.0:1789760731.736687:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508310144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760731.736735:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20164608, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:0.0:1789760731.736745:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54dac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.736768:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:0.0:1789760731.736773:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760731.736780:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54dac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760731.736793:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54dac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760731.814948:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972243 is committed 00002000:00010000:1.0:1789760731.815228:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20164608 -> 20168704 00010000:00010000:1.0:1789760731.815243:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54dac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.815266:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54dac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.815276:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54dac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.815282:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760731.815323:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54dac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.815331:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54dac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.815350:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54dac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.815359:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54dac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20164608->20168703] (req 20164608->20168703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.815454:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508310144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 78763us (79024us total) trans 4294972243 rc 0/0 00000100:00100000:1.0:1789760731.815468:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4955 00000100:00100000:1.0:1789760731.815477:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710769220819 time 710841385504 state 0x2 ref 0 00000100:00100000:0.0:1789760731.824254:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760731.824876:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508310272 00000100:00100000:3.0:1789760731.824886:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508310272 00000100:00100000:3.0:1789760731.824947:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710769220819 time 710841385504 state 0x0 ref 1 00000100:00100000:3.0:1789760731.824963:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710841645920 time 710841385504 state 0x1 ref 1 00000100:00100000:3.0:1789760731.825000:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710841645920 time 710929710351 state 0x0 ref 1 00000100:00100000:3.0:1789760731.825014:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508310272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760731.825062:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20168704, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:3.0:1789760731.825070:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54db3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.825084:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760731.825089:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.825095:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54db3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.825109:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54db3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760731.874855:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972244 is committed 00002000:00010000:1.0:1789760731.875342:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20168704 -> 20172800 00010000:00010000:1.0:1789760731.875357:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54db3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.875370:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54db3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.875380:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54db3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.875388:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760731.875392:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54db3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.875400:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54db3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.875421:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54db3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760731.875430:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe54db3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20168704->20172799] (req 20168704->20172799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760731.875552:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508310272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50535us (51301us total) trans 4294972244 rc 0/0 00000100:00100000:1.0:1789760731.875563:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4956 00000100:00100000:1.0:1789760731.875573:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710841645920 time 710929710351 state 0x2 ref 0 00000100:00100000:2.0:1789760731.879731:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760731.879896:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508310400 00000100:00100000:3.0:1789760731.879906:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508310400 00000100:00100000:3.0:1789760731.879961:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710841645920 time 710929710351 state 0x0 ref 1 00000100:00100000:3.0:1789760731.879975:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710929970767 time 710929710351 state 0x1 ref 1 00000100:00100000:3.0:1789760731.880005:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710929970767 time 710984715866 state 0x0 ref 1 00000100:00100000:3.0:1789760731.880019:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508310400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760731.880065:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20172800, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:3.0:1789760731.880074:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54dba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.880088:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760731.880092:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760731.880098:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54dba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760731.880112:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54dba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.904434:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760731.994317:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972245 is committed 00002000:00010000:2.0:1789760731.994455:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20172800 -> 20176896 00010000:00010000:2.0:1789760731.994470:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54dba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.994482:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54dba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.994502:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54dba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.994511:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760731.994516:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54dba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.994525:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54dba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.994545:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54dba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760731.994554:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54dba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20172800->20176895] (req 20172800->20176895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760731.994645:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508310400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 114624us (114920us total) trans 4294972245 rc 0/0 00000100:00100000:2.0:1789760731.994656:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4957 00000100:00100000:2.0:1789760731.994666:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710929970767 time 710984715866 state 0x2 ref 0 00000100:00100000:3.0:1789760732.006087:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.006190:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508310528 00000100:00100000:1.0:1789760732.006201:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508310528 00000100:00100000:1.0:1789760732.006255:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710929970767 time 710984715866 state 0x0 ref 1 00000100:00100000:1.0:1789760732.006270:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710984976282 time 710984715866 state 0x1 ref 1 00000100:00100000:1.0:1789760732.006334:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710984976282 time 711111044148 state 0x0 ref 1 00000100:00100000:1.0:1789760732.006346:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508310528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.006393:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20176896, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:1.0:1789760732.006402:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54dc1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.006419:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.006423:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.006429:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54dc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.006442:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54dc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.030585:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534066304:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760732.030656:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760732.030731:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534066304 00000100:00100000:3.0:1789760732.030738:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534066304 00000100:00100000:3.0:1789760732.030781:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 424 00000100:00100000:3.0:1789760732.030791:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3473:x1876699534066304:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760732.030828:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789760732.030873:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3473:x1876699534066304:12345-0@lo:41:osp-pre-0-1.0 Request processed in 83us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760732.030880:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 424 00000100:00100000:3.0:1789760732.030923:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534066304:0@lo:41:osp-pre-0-1.0 00000001:00080000:3.0:1789760732.052676:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972246 is committed 00002000:00010000:2.0:1789760732.052858:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20176896 -> 20180992 00010000:00010000:2.0:1789760732.052874:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54dc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.052886:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54dc1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.052900:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54dc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.052908:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760732.052912:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54dc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.052921:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54dc1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.052940:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54dc1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.052948:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54dc1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20176896->20180991] (req 20176896->20180991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.053050:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508310528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46701us (46977us total) trans 4294972246 rc 0/0 00000100:00100000:2.0:1789760732.053073:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4958 00000100:00100000:2.0:1789760732.053084:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710984976282 time 711111044148 state 0x2 ref 0 00000100:00100000:0.0:1789760732.062634:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.066468:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508310656 00000100:00100000:1.0:1789760732.066483:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508310656 00000100:00100000:1.0:1789760732.066545:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 710984976282 time 711111044148 state 0x0 ref 1 00000100:00100000:1.0:1789760732.066556:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711111304564 time 711111044148 state 0x1 ref 1 00000100:00100000:1.0:1789760732.066578:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711111304564 time 711171289151 state 0x0 ref 1 00000100:00100000:1.0:1789760732.066587:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508310656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.066625:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20180992, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:1.0:1789760732.066631:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54dc8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.066644:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.066647:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.066651:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54dc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.066661:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54dc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.115032:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972247 is committed 00002000:00010000:1.0:1789760732.115259:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20180992 -> 20185088 00010000:00010000:1.0:1789760732.115274:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54dc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.115363:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54dc8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.115371:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54dc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.115378:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760732.115382:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54dc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.115390:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54dc8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.115409:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54dc8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.115416:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54dc8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20180992->20185087] (req 20180992->20185087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.115597:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508310656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49006us (52968us total) trans 4294972247 rc 0/0 00000100:00100000:1.0:1789760732.115607:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4959 00000100:00100000:1.0:1789760732.115617:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711111304564 time 711171289151 state 0x2 ref 0 00000100:00100000:3.0:1789760732.118589:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760732.118799:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508310784 00000100:00100000:2.0:1789760732.118810:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508310784 00000100:00100000:2.0:1789760732.118863:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711111304564 time 711171289151 state 0x0 ref 1 00000100:00100000:2.0:1789760732.118877:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711171549567 time 711171289151 state 0x1 ref 1 00000100:00100000:2.0:1789760732.118908:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711171549567 time 711223619013 state 0x0 ref 1 00000100:00100000:2.0:1789760732.118919:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508310784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760732.118966:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20185088, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:2.0:1789760732.118974:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54dcf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.118989:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760732.118993:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.118998:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54dcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.119011:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54dcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.167773:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972248 is committed 00002000:00010000:0.0:1789760732.167823:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20185088 -> 20189184 00010000:00010000:0.0:1789760732.167840:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54dcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.167853:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54dcf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.167862:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54dcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.167870:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760732.167875:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54dcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.167885:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54dcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.167907:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54dcf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.167914:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54dcf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20185088->20189183] (req 20185088->20189183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.168004:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508310784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49081us (49419us total) trans 4294972248 rc 0/0 00000100:00100000:0.0:1789760732.168015:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4960 00000100:00100000:0.0:1789760732.168025:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711171549567 time 711223619013 state 0x2 ref 0 00000100:00100000:3.0:1789760732.176670:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760732.176816:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508310912 00000100:00100000:2.0:1789760732.176827:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508310912 00000100:00100000:2.0:1789760732.176882:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711171549567 time 711223619013 state 0x0 ref 1 00000100:00100000:2.0:1789760732.176896:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711223879429 time 711223619013 state 0x1 ref 1 00000100:00100000:2.0:1789760732.176938:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711223879429 time 711281638925 state 0x0 ref 1 00000100:00100000:2.0:1789760732.176951:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508310912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760732.176999:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20189184, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:2.0:1789760732.177008:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54dd6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.177024:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:2.0:1789760732.177028:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760732.177034:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54dd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.177049:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54dd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760732.213335:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20189184 -> 20193280 00010000:00010000:3.0:1789760732.213354:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54dd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.213367:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54dd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.213377:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54dd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.213391:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760732.213396:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54dd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.213406:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54dd6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.213425:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54dd6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.213432:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54dd6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20189184->20193279] (req 20189184->20193279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.213637:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508310912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36683us (36974us total) trans 4294972249 rc 0/0 00000100:00100000:3.0:1789760732.213647:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4961 00000100:00100000:3.0:1789760732.213657:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711223879429 time 711281638925 state 0x2 ref 0 00000001:00080000:3.0:1789760732.213693:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972249 is committed 00000100:00100000:2.0:1789760732.219049:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.219139:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508311040 00000100:00100000:1.0:1789760732.219161:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508311040 00000100:00100000:1.0:1789760732.219215:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711223879429 time 711281638925 state 0x0 ref 1 00000100:00100000:1.0:1789760732.219229:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711281899341 time 711281638925 state 0x1 ref 1 00000100:00100000:1.0:1789760732.219259:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711281899341 time 711323969808 state 0x0 ref 1 00000100:00100000:1.0:1789760732.219269:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508311040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.219341:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20193280, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:1.0:1789760732.219350:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54ddd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.219366:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.219371:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.219377:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54ddd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.219391:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54ddd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.256675:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972250 is committed 00002000:00010000:3.0:1789760732.256802:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20193280 -> 20197376 00010000:00010000:3.0:1789760732.256818:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54ddd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.256831:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54ddd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.256841:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54ddd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.256846:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760732.256849:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54ddd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.256855:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54ddd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.256870:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54ddd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.256875:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe54ddd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20193280->20197375] (req 20193280->20197375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760732.256959:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508311040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37685us (37915us total) trans 4294972250 rc 0/0 00000100:00100000:3.0:1789760732.256968:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4962 00000100:00100000:3.0:1789760732.256977:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711281899341 time 711323969808 state 0x2 ref 0 00000100:00100000:2.0:1789760732.259914:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760732.260029:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508311168 00000100:00100000:0.0:1789760732.260038:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508311168 00000100:00100000:0.0:1789760732.260166:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711281899341 time 711323969808 state 0x0 ref 1 00000100:00100000:0.0:1789760732.260181:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711324230224 time 711323969808 state 0x1 ref 1 00000100:00100000:0.0:1789760732.260209:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711324230224 time 711364919615 state 0x0 ref 1 00000100:00100000:0.0:1789760732.260223:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508311168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760732.260271:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20197376, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:0.0:1789760732.260280:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54de4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.260329:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760732.260335:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.260341:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54de4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.260355:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54de4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760732.296226:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20197376 -> 20201472 00010000:00010000:1.0:1789760732.296246:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54de4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.296258:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54de4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.296272:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54de4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.296279:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760732.296311:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54de4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.296320:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54de4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.296339:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54de4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.296346:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54de4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20197376->20201471] (req 20197376->20201471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.296433:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508311168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36207us (36524us total) trans 4294972251 rc 0/0 00000100:00100000:1.0:1789760732.296444:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4963 00000100:00100000:1.0:1789760732.296453:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711324230224 time 711364919615 state 0x2 ref 0 00000001:00080000:1.0:1789760732.296998:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972251 is committed 00000100:00100000:2.0:1789760732.304714:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760732.304810:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508311296 00000100:00100000:3.0:1789760732.304821:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508311296 00000100:00100000:3.0:1789760732.304874:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711324230224 time 711364919615 state 0x0 ref 1 00000100:00100000:3.0:1789760732.304889:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711365180031 time 711364919615 state 0x1 ref 1 00000100:00100000:3.0:1789760732.304965:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711365180031 time 711409675180 state 0x0 ref 1 00000100:00100000:3.0:1789760732.304977:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508311296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760732.305025:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20201472, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:3.0:1789760732.305034:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54deb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.305049:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760732.305054:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.305060:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54deb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.305074:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54deb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.344863:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972252 is committed 00002000:00010000:1.0:1789760732.344944:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20201472 -> 20205568 00010000:00010000:1.0:1789760732.344959:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54deb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.344978:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54deb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.344987:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54deb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.344995:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760732.345000:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54deb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.345008:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54deb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.345027:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54deb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.345035:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54deb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20201472->20205567] (req 20201472->20205567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.345585:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508311296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40604us (40874us total) trans 4294972252 rc 0/0 00000100:00100000:1.0:1789760732.345597:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4964 00000100:00100000:1.0:1789760732.345607:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711365180031 time 711409675180 state 0x2 ref 0 00000100:00100000:0.0:1789760732.350905:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760732.350977:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508311424 00000100:00100000:3.0:1789760732.351022:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508311424 00000100:00100000:3.0:1789760732.351073:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711365180031 time 711409675180 state 0x0 ref 1 00000100:00100000:3.0:1789760732.351087:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711409935596 time 711409675180 state 0x1 ref 1 00000100:00100000:3.0:1789760732.351117:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711409935596 time 711455827497 state 0x0 ref 1 00000100:00100000:3.0:1789760732.351134:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508311424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760732.351256:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20205568, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:3.0:1789760732.351266:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54df2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.351283:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760732.351383:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760732.351391:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54df2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760732.351406:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54df2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760732.403257:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20205568 -> 20209664 00010000:00010000:1.0:1789760732.403278:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54df2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.403315:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54df2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.403325:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54df2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.403341:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760732.403346:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54df2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.403356:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54df2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.403382:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54df2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.403390:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54df2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20205568->20209663] (req 20205568->20209663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.403475:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508311424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52339us (52576us total) trans 4294972253 rc 0/0 00000100:00100000:1.0:1789760732.403485:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4965 00000100:00100000:1.0:1789760732.403626:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711409935596 time 711455827497 state 0x2 ref 0 00000001:00080000:0.0:1789760732.403760:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972253 is committed 00000100:00100000:3.0:1789760732.407101:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.407198:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508311552 00000100:00100000:1.0:1789760732.407209:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508311552 00000100:00100000:1.0:1789760732.407263:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711409935596 time 711455827497 state 0x0 ref 1 00000100:00100000:1.0:1789760732.407278:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711456087913 time 711455827497 state 0x1 ref 1 00000100:00100000:1.0:1789760732.407327:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711456087913 time 711512037346 state 0x0 ref 1 00000100:00100000:1.0:1789760732.407349:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508311552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.407406:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20209664, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:1.0:1789760732.407414:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54df9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.407430:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.407434:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.407442:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54df9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.407456:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54df9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760732.449237:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20209664 -> 20213760 00010000:00010000:1.0:1789760732.449256:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54df9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.449269:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54df9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.449278:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54df9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.449314:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760732.449320:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54df9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.449329:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54df9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.449346:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54df9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.449353:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54df9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20209664->20213759] (req 20209664->20213759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.449439:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508311552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42087us (42342us total) trans 4294972254 rc 0/0 00000100:00100000:1.0:1789760732.449449:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4966 00000100:00100000:1.0:1789760732.449458:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711456087913 time 711512037346 state 0x2 ref 0 00000001:00080000:0.0:1789760732.449599:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972254 is committed 00000100:00100000:0.0:1789760732.452826:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.452984:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508311680 00000100:00100000:1.0:1789760732.452994:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508311680 00000100:00100000:1.0:1789760732.453046:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711456087913 time 711512037346 state 0x0 ref 1 00000100:00100000:1.0:1789760732.453072:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711512297762 time 711512037346 state 0x1 ref 1 00000100:00100000:1.0:1789760732.453103:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711512297762 time 711557813830 state 0x0 ref 1 00000100:00100000:1.0:1789760732.453115:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508311680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.453179:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20213760, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:1.0:1789760732.453190:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54e00 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.453203:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.453208:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.453214:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54e00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.453230:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54e00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.475610:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972255 is committed 00002000:00010000:0.0:1789760732.475770:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20213760 -> 20217856 00010000:00010000:0.0:1789760732.475784:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54e00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.475798:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54e00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.475807:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54e00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.475814:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760732.475818:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54e00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.475827:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54e00 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.475854:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54e00 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.475862:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54e00 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20213760->20217855] (req 20213760->20217855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.475953:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508311680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22834us (23133us total) trans 4294972255 rc 0/0 00000100:00100000:0.0:1789760732.475963:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4967 00000100:00100000:0.0:1789760732.475972:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711512297762 time 711557813830 state 0x2 ref 0 00000100:00100000:2.0:1789760732.479759:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.480340:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508311808 00000100:00100000:1.0:1789760732.480349:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508311808 00000100:00100000:1.0:1789760732.480407:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711512297762 time 711557813830 state 0x0 ref 1 00000100:00100000:1.0:1789760732.480422:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711558074246 time 711557813830 state 0x1 ref 1 00000100:00100000:1.0:1789760732.480447:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711558074246 time 711585158123 state 0x0 ref 1 00000100:00100000:1.0:1789760732.480459:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508311808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.480789:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20217856, mtime: 0x6aad94dc, atime: 0x0, ctime: 0x6aad94dc, blocks: 0xa007 00010000:00010000:1.0:1789760732.480799:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54e07 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.480815:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.480819:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.480824:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54e07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.480837:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54e07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760732.506907:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20217856 -> 20221952 00002000:00010000:0.0:1789760732.506916:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760732 -> 1789760733 00002000:00010000:0.0:1789760732.506919:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760732 -> 1789760733 00010000:00010000:0.0:1789760732.506932:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54e07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.506945:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54e07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.506954:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54e07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.506962:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760732.506966:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54e07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.506976:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54e07 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.506997:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54e07 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.507004:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54e07 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20217856->20221951] (req 20217856->20221951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.507087:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508311808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26625us (27334us total) trans 4294972256 rc 0/0 00000100:00100000:0.0:1789760732.507097:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4968 00000100:00100000:0.0:1789760732.507108:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711558074246 time 711585158123 state 0x2 ref 0 00000001:00080000:1.0:1789760732.507256:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972256 is committed 00000100:00100000:3.0:1789760732.509982:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.510085:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508311936 00000100:00100000:1.0:1789760732.510094:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508311936 00000100:00100000:1.0:1789760732.510156:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711558074246 time 711585158123 state 0x0 ref 1 00000100:00100000:1.0:1789760732.510170:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711585418539 time 711585158123 state 0x1 ref 1 00000100:00100000:1.0:1789760732.510196:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711585418539 time 711614906461 state 0x0 ref 1 00000100:00100000:1.0:1789760732.510207:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508311936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.510253:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20221952, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:1.0:1789760732.510262:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.510278:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.510313:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.510321:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.510336:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760732.535641:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20221952 -> 20226048 00010000:00010000:1.0:1789760732.535662:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.535676:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.535685:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.535692:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760732.535697:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.535706:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.535725:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e0e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.535733:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20221952->20226047] (req 20221952->20226047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.535841:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508311936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25625us (25859us total) trans 4294972257 rc 0/0 00000100:00100000:1.0:1789760732.535853:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4969 00000100:00100000:1.0:1789760732.535864:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711585418539 time 711614906461 state 0x2 ref 0 00000001:00080000:0.0:1789760732.535953:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972257 is committed 00000100:00100000:2.0:1789760732.541257:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760732.541395:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508312064 00000100:00100000:0.0:1789760732.541405:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508312064 00000100:00100000:0.0:1789760732.541461:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711585418539 time 711614906461 state 0x0 ref 1 00000100:00100000:0.0:1789760732.541487:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711615166877 time 711614906461 state 0x1 ref 1 00000100:00100000:0.0:1789760732.541520:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711615166877 time 711646230608 state 0x0 ref 1 00000100:00100000:0.0:1789760732.541534:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508312064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760732.541581:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20226048, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:0.0:1789760732.541590:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.541605:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760732.541610:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.541617:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.541631:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760732.578769:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972258 is committed 00002000:00010000:0.0:1789760732.578829:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20226048 -> 20230144 00010000:00010000:0.0:1789760732.578843:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.578856:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.578865:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.578871:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760732.578877:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.578885:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.578906:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.578914:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20226048->20230143] (req 20226048->20230143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.579005:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508312064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37468us (37753us total) trans 4294972258 rc 0/0 00000100:00100000:0.0:1789760732.579015:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4970 00000100:00100000:0.0:1789760732.579024:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711615166877 time 711646230608 state 0x2 ref 0 00000100:00100000:3.0:1789760732.582831:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760732.582943:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508312192 00000100:00100000:0.0:1789760732.582953:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508312192 00000100:00100000:0.0:1789760732.583003:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711615166877 time 711646230608 state 0x0 ref 1 00000100:00100000:0.0:1789760732.583017:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711646491024 time 711646230608 state 0x1 ref 1 00000100:00100000:0.0:1789760732.583045:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711646491024 time 711687756169 state 0x0 ref 1 00000100:00100000:0.0:1789760732.583065:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508312192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760732.583111:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20230144, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:0.0:1789760732.583120:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54e1c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.583135:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760732.583149:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.583156:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54e1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.583170:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54e1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760732.628632:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20230144 -> 20234240 00010000:00010000:2.0:1789760732.628651:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54e1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.628663:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54e1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.628672:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54e1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.628679:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760732.628683:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54e1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.628692:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54e1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.628709:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54e1c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.628716:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54e1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20230144->20234239] (req 20230144->20234239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.628809:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508312192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45750us (45984us total) trans 4294972259 rc 0/0 00000100:00100000:2.0:1789760732.628818:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4971 00000100:00100000:2.0:1789760732.628828:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711646491024 time 711687756169 state 0x2 ref 0 00000001:00080000:0.0:1789760732.628916:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972259 is committed 00000100:00100000:3.0:1789760732.636405:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.638394:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508312320 00000100:00100000:1.0:1789760732.638464:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508312320 00000100:00100000:1.0:1789760732.638530:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711646491024 time 711687756169 state 0x0 ref 1 00000100:00100000:1.0:1789760732.638549:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711688016585 time 711687756169 state 0x1 ref 1 00000100:00100000:1.0:1789760732.638773:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711688016585 time 711743481751 state 0x0 ref 1 00000100:00100000:1.0:1789760732.638787:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508312320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.638837:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20234240, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:1.0:1789760732.638846:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e23 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.638862:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.638867:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.638873:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.638887:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.660476:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972260 is committed 00002000:00010000:1.0:1789760732.660639:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20234240 -> 20238336 00010000:00010000:1.0:1789760732.660653:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.660666:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.660676:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.660684:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760732.660689:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.660697:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e23 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.660716:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e23 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.660723:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20234240->20238335] (req 20234240->20238335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.660839:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508312320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22050us (24439us total) trans 4294972260 rc 0/0 00000100:00100000:1.0:1789760732.660852:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4972 00000100:00100000:1.0:1789760732.660863:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711688016585 time 711743481751 state 0x2 ref 0 00000100:00100000:3.0:1789760732.665752:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760732.665846:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508312448 00000100:00100000:0.0:1789760732.665856:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508312448 00000100:00100000:0.0:1789760732.665909:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711688016585 time 711743481751 state 0x0 ref 1 00000100:00100000:0.0:1789760732.665924:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711743742167 time 711743481751 state 0x1 ref 1 00000100:00100000:0.0:1789760732.665952:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711743742167 time 711770662584 state 0x0 ref 1 00000100:00100000:0.0:1789760732.665964:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508312448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760732.666013:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20238336, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:0.0:1789760732.666023:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.666037:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760732.666041:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.666047:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.666062:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760732.705062:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20238336 -> 20242432 00010000:00010000:1.0:1789760732.705081:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.705094:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.705103:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.705109:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760732.705114:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.705121:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.705151:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e2a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.705159:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54e2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20238336->20242431] (req 20238336->20242431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.705262:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508312448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39294us (39512us total) trans 4294972261 rc 0/0 00000100:00100000:1.0:1789760732.705272:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4973 00000100:00100000:1.0:1789760732.705281:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711743742167 time 711770662584 state 0x2 ref 0 00000001:00080000:2.0:1789760732.705380:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972261 is committed 00000100:00100000:3.0:1789760732.708585:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760732.708708:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508312576 00000100:00100000:0.0:1789760732.708718:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508312576 00000100:00100000:0.0:1789760732.708767:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711743742167 time 711770662584 state 0x0 ref 1 00000100:00100000:0.0:1789760732.708782:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711770923000 time 711770662584 state 0x1 ref 1 00000100:00100000:0.0:1789760732.708809:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711770923000 time 711813519366 state 0x0 ref 1 00000100:00100000:0.0:1789760732.708820:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508312576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760732.708867:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20242432, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:0.0:1789760732.708877:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54e31 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.708892:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760732.708897:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.708903:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54e31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.708917:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54e31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760732.740469:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972262 is committed 00002000:00010000:0.0:1789760732.740664:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20242432 -> 20246528 00010000:00010000:0.0:1789760732.740679:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54e31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.740697:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54e31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.740705:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54e31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.740712:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760732.740716:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54e31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.740725:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54e31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.740743:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54e31 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.740750:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe54e31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20242432->20246527] (req 20242432->20246527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.740835:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508312576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32012us (32255us total) trans 4294972262 rc 0/0 00000100:00100000:0.0:1789760732.740844:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4974 00000100:00100000:0.0:1789760732.740854:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711770923000 time 711813519366 state 0x2 ref 0 00000100:00100000:0.0:1789760732.744870:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.745182:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508312704 00000100:00100000:1.0:1789760732.745192:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508312704 00000100:00100000:1.0:1789760732.745243:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711770923000 time 711813519366 state 0x0 ref 1 00000100:00100000:1.0:1789760732.745258:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711813779782 time 711813519366 state 0x1 ref 1 00000100:00100000:1.0:1789760732.745313:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711813779782 time 711849996429 state 0x0 ref 1 00000100:00100000:1.0:1789760732.745326:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508312704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.745370:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20246528, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:1.0:1789760732.745379:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.745395:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.745400:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.745406:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.745419:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.768821:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972263 is committed 00002000:00010000:1.0:1789760732.768917:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20246528 -> 20250624 00010000:00010000:1.0:1789760732.768932:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.768945:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.768954:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.768962:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760732.768966:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.768976:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.768994:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e38 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.769002:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20246528->20250623] (req 20246528->20250623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.769182:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508312704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23853us (24316us total) trans 4294972263 rc 0/0 00000100:00100000:1.0:1789760732.769196:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4975 00000100:00100000:1.0:1789760732.769207:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711813779782 time 711849996429 state 0x2 ref 0 00000100:00100000:2.0:1789760732.772660:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.772748:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508312832 00000100:00100000:1.0:1789760732.772758:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508312832 00000100:00100000:1.0:1789760732.772810:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711813779782 time 711849996429 state 0x0 ref 1 00000100:00100000:1.0:1789760732.772826:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711850256845 time 711849996429 state 0x1 ref 1 00000100:00100000:1.0:1789760732.772888:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711850256845 time 711877598001 state 0x0 ref 1 00000100:00100000:1.0:1789760732.772901:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508312832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.772947:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20250624, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:1.0:1789760732.772956:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e3f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.772972:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.772977:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.772983:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.772998:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.799275:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972264 is committed 00002000:00010000:1.0:1789760732.799335:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20250624 -> 20254720 00010000:00010000:1.0:1789760732.799350:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.799363:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.799371:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.799377:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760732.799382:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.799390:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.799480:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e3f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.799488:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20250624->20254719] (req 20250624->20254719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.799684:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508312832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26776us (27024us total) trans 4294972264 rc 0/0 00000100:00100000:1.0:1789760732.799694:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4976 00000100:00100000:1.0:1789760732.799703:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711850256845 time 711877598001 state 0x2 ref 0 00000100:00100000:2.0:1789760732.803259:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760732.803422:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508312960 00000100:00100000:0.0:1789760732.803432:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508312960 00000100:00100000:0.0:1789760732.803482:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711850256845 time 711877598001 state 0x0 ref 1 00000100:00100000:0.0:1789760732.803561:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711877858417 time 711877598001 state 0x1 ref 1 00000100:00100000:0.0:1789760732.803630:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711877858417 time 711908339650 state 0x0 ref 1 00000100:00100000:0.0:1789760732.803643:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508312960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760732.803687:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20254720, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:0.0:1789760732.803696:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54e46 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.803711:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:0.0:1789760732.803715:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.803722:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54e46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.803735:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54e46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760732.831353:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972265 is committed 00002000:00010000:0.0:1789760732.831410:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20254720 -> 20258816 00010000:00010000:0.0:1789760732.831424:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54e46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.831437:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54e46 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.831452:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54e46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.831459:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760732.831464:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54e46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.831473:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54e46 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.831544:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54e46 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.831552:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54e46 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20254720->20258815] (req 20254720->20258815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.831644:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508312960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27999us (28390us total) trans 4294972265 rc 0/0 00000100:00100000:0.0:1789760732.831654:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4977 00000100:00100000:0.0:1789760732.831663:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711877858417 time 711908339650 state 0x2 ref 0 00000100:00100000:0.0:1789760732.834279:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.834572:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508313088 00000100:00100000:1.0:1789760732.834583:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508313088 00000100:00100000:1.0:1789760732.834634:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711877858417 time 711908339650 state 0x0 ref 1 00000100:00100000:1.0:1789760732.834649:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711908600066 time 711908339650 state 0x1 ref 1 00000100:00100000:1.0:1789760732.834679:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711908600066 time 711939390344 state 0x0 ref 1 00000100:00100000:1.0:1789760732.834692:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508313088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.834737:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20258816, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:1.0:1789760732.834745:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e4d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.834759:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.834762:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.834769:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.834783:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760732.862872:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20258816 -> 20262912 00010000:00010000:1.0:1789760732.862890:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.862903:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.862916:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.862923:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760732.862927:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.862934:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.862952:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e4d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.862958:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20258816->20262911] (req 20258816->20262911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760732.863048:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508313088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28354us (28775us total) trans 4294972266 rc 0/0 00000100:00100000:1.0:1789760732.863067:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4978 00000100:00100000:1.0:1789760732.863076:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711908600066 time 711939390344 state 0x2 ref 0 00000001:00080000:3.0:1789760732.864963:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972266 is committed 00000100:00100000:2.0:1789760732.867947:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.868029:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508313216 00000100:00100000:1.0:1789760732.868038:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508313216 00000100:00100000:1.0:1789760732.868090:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711908600066 time 711939390344 state 0x0 ref 1 00000100:00100000:1.0:1789760732.868105:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711939650760 time 711939390344 state 0x1 ref 1 00000100:00100000:1.0:1789760732.868135:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711939650760 time 711972845880 state 0x0 ref 1 00000100:00100000:1.0:1789760732.868158:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508313216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.868205:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20262912, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:1.0:1789760732.868214:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.868229:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=711s v=(0 0) 00010000:00010000:1.0:1789760732.868234:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.868240:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.868254:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760732.895693:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972267 is committed 00002000:00010000:2.0:1789760732.895873:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20262912 -> 20267008 00010000:00010000:2.0:1789760732.895889:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.895908:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.895917:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.895923:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760732.895928:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.895936:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.895956:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760732.895964:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54e54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20262912->20267007] (req 20262912->20267007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760732.896095:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508313216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27934us (28161us total) trans 4294972267 rc 0/0 00000100:00100000:2.0:1789760732.896105:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4979 00000100:00100000:2.0:1789760732.896115:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711939650760 time 711972845880 state 0x2 ref 0 00000100:00100000:1.0:1789760732.909395:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760732.914714:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508313344 00000100:00100000:0.0:1789760732.914731:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508313344 00000100:00100000:0.0:1789760732.914804:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711939650760 time 711972845880 state 0x0 ref 1 00000100:00100000:0.0:1789760732.914824:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711973106296 time 711972845880 state 0x1 ref 1 00000100:00100000:0.0:1789760732.914856:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711973106296 time 712019566902 state 0x0 ref 1 00000100:00100000:0.0:1789760732.914869:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508313344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760732.914916:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20267008, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:0.0:1789760732.914926:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.914943:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760732.914947:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760732.914954:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.914970:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.927854:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760732.966976:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972268 is committed 00002000:00010000:0.0:1789760732.967168:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20267008 -> 20271104 00010000:00010000:0.0:1789760732.967185:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.967197:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.967206:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.967220:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760732.967225:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.967234:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e5b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.967253:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e5b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760732.967260:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20267008->20271103] (req 20267008->20271103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760732.967406:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508313344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52534us (58026us total) trans 4294972268 rc 0/0 00000100:00100000:0.0:1789760732.967417:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4980 00000100:00100000:0.0:1789760732.967427:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711973106296 time 712019566902 state 0x2 ref 0 00000100:00100000:2.0:1789760732.976423:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760732.976675:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508313472 00000100:00100000:1.0:1789760732.976685:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508313472 00000100:00100000:1.0:1789760732.976739:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 711973106296 time 712019566902 state 0x0 ref 1 00000100:00100000:1.0:1789760732.976754:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712019827318 time 712019566902 state 0x1 ref 1 00000100:00100000:1.0:1789760732.976784:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712019827318 time 712081494720 state 0x0 ref 1 00000100:00100000:1.0:1789760732.976797:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508313472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760732.976844:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20271104, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:1.0:1789760732.976853:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.976869:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760732.976873:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760732.976878:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760732.976890:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760733.002849:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972269 is committed 00002000:00010000:1.0:1789760733.003018:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20271104 -> 20275200 00010000:00010000:1.0:1789760733.003034:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.003047:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.003057:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.003064:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760733.003069:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.003079:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.003100:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e62 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.003109:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20271104->20275199] (req 20271104->20275199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.003218:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508313472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26419us (26810us total) trans 4294972269 rc 0/0 00000100:00100000:1.0:1789760733.003229:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4981 00000100:00100000:1.0:1789760733.003239:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712019827318 time 712081494720 state 0x2 ref 0 00000100:00100000:2.0:1789760733.007901:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760733.009533:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508313600 00000100:00100000:1.0:1789760733.009545:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508313600 00000100:00100000:1.0:1789760733.009624:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712019827318 time 712081494720 state 0x0 ref 1 00000100:00100000:1.0:1789760733.009640:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712081755136 time 712081494720 state 0x1 ref 1 00000100:00100000:1.0:1789760733.009664:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712081755136 time 712114375374 state 0x0 ref 1 00000100:00100000:1.0:1789760733.009678:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508313600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760733.009722:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20275200, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:1.0:1789760733.009731:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.009747:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760733.009751:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.009757:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.009771:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760733.010791:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760733.033832:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972270 is committed 00002000:00010000:1.0:1789760733.033972:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20275200 -> 20279296 00010000:00010000:1.0:1789760733.033989:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.034001:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.034015:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.034023:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760733.034027:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.034036:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.034065:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.034212:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54e69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20275200->20279295] (req 20275200->20279295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.034375:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508313600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24696us (26477us total) trans 4294972270 rc 0/0 00000100:00100000:1.0:1789760733.034386:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4982 00000100:00100000:1.0:1789760733.034396:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712081755136 time 712114375374 state 0x2 ref 0 00000100:00100000:3.0:1789760733.042607:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760733.042771:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508313728 00000100:00100000:0.0:1789760733.042780:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508313728 00000100:00100000:0.0:1789760733.042836:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712081755136 time 712114375374 state 0x0 ref 1 00000100:00100000:0.0:1789760733.042851:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712114635790 time 712114375374 state 0x1 ref 1 00000100:00100000:0.0:1789760733.042914:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712114635790 time 712147624241 state 0x0 ref 1 00000100:00100000:0.0:1789760733.042930:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508313728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760733.042976:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20279296, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:0.0:1789760733.042984:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.042999:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760733.043004:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.043010:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.043023:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.074615:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972271 is committed 00002000:00010000:3.0:1789760733.074774:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20279296 -> 20283392 00010000:00010000:3.0:1789760733.074788:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.074801:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.074810:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.074817:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760733.074822:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.074832:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e70 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.074851:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e70 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.074860:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20279296->20283391] (req 20279296->20283391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.074957:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508313728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32023us (32357us total) trans 4294972271 rc 0/0 00000100:00100000:3.0:1789760733.074968:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4983 00000100:00100000:3.0:1789760733.074979:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712114635790 time 712147624241 state 0x2 ref 0 00000100:00100000:2.0:1789760733.081958:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760733.084359:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508313856 00000100:00100000:1.0:1789760733.084372:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508313856 00000100:00100000:1.0:1789760733.084431:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712114635790 time 712147624241 state 0x0 ref 1 00000100:00100000:1.0:1789760733.084446:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712147884657 time 712147624241 state 0x1 ref 1 00000100:00100000:1.0:1789760733.084529:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712147884657 time 712189239387 state 0x0 ref 1 00000100:00100000:1.0:1789760733.084543:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508313856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760733.084600:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20283392, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:1.0:1789760733.084609:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54e77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.084624:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760733.084629:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.084635:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54e77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.084649:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54e77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.127030:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972272 is committed 00002000:00010000:2.0:1789760733.127257:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20283392 -> 20287488 00010000:00010000:2.0:1789760733.127274:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54e77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.127310:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54e77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.127319:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54e77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.127327:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760733.127332:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54e77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.127341:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54e77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.127360:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54e77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.127368:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe54e77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20283392->20287487] (req 20283392->20287487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.127457:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508313856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42913us (45505us total) trans 4294972272 rc 0/0 00000100:00100000:2.0:1789760733.127472:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4984 00000100:00100000:2.0:1789760733.127482:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712147884657 time 712189239387 state 0x2 ref 0 00000100:00100000:3.0:1789760733.130395:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760733.130792:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508313984 00000100:00100000:2.0:1789760733.130803:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508313984 00000100:00100000:2.0:1789760733.130857:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712147884657 time 712189239387 state 0x0 ref 1 00000100:00100000:2.0:1789760733.130872:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712189499803 time 712189239387 state 0x1 ref 1 00000100:00100000:2.0:1789760733.130903:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712189499803 time 712235612986 state 0x0 ref 1 00000100:00100000:2.0:1789760733.130915:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508313984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760733.130964:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20287488, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:2.0:1789760733.130974:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54e7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.130989:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760733.130994:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760733.131001:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54e7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.131015:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54e7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.160308:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972273 is committed 00002000:00010000:3.0:1789760733.160560:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20287488 -> 20291584 00010000:00010000:3.0:1789760733.160575:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54e7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.160587:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54e7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.160595:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54e7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.160601:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760733.160606:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54e7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.160613:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54e7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.160676:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54e7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.160684:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe54e7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20287488->20291583] (req 20287488->20291583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.160781:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508313984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29861us (30387us total) trans 4294972273 rc 0/0 00000100:00100000:3.0:1789760733.160792:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4985 00000100:00100000:3.0:1789760733.160803:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712189499803 time 712235612986 state 0x2 ref 0 00000100:00100000:3.0:1789760733.163933:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760733.164523:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508314112 00000100:00100000:0.0:1789760733.164535:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508314112 00000100:00100000:0.0:1789760733.164588:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712189499803 time 712235612986 state 0x0 ref 1 00000100:00100000:0.0:1789760733.164602:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712235873402 time 712235612986 state 0x1 ref 1 00000100:00100000:0.0:1789760733.164632:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712235873402 time 712269343198 state 0x0 ref 1 00000100:00100000:0.0:1789760733.164644:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508314112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760733.164690:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20291584, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:0.0:1789760733.164699:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54e85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.164714:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760733.164719:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.164725:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54e85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.164738:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54e85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760733.207688:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20291584 -> 20295680 00010000:00010000:2.0:1789760733.207710:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54e85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.207722:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54e85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.207732:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54e85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.207739:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760733.207744:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54e85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.207753:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54e85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.207773:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54e85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.207781:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe54e85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20291584->20295679] (req 20291584->20295679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.207866:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508314112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43220us (43939us total) trans 4294972274 rc 0/0 00000100:00100000:2.0:1789760733.207874:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4986 00000100:00100000:2.0:1789760733.207882:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712235873402 time 712269343198 state 0x2 ref 0 00000001:00080000:2.0:1789760733.207919:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972274 is committed 00000100:00100000:2.0:1789760733.211216:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760733.211326:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508314240 00000100:00100000:1.0:1789760733.211336:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508314240 00000100:00100000:1.0:1789760733.211393:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712235873402 time 712269343198 state 0x0 ref 1 00000100:00100000:1.0:1789760733.211409:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712269603614 time 712269343198 state 0x1 ref 1 00000100:00100000:1.0:1789760733.211479:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712269603614 time 712316189486 state 0x0 ref 1 00000100:00100000:1.0:1789760733.211506:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d45c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508314240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760733.211552:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20295680, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:1.0:1789760733.211561:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.211578:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760733.211582:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.211588:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.211602:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.269675:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972275 is committed 00002000:00010000:1.0:1789760733.271127:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20295680 -> 20299776 00010000:00010000:1.0:1789760733.271159:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.271171:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.271179:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.271186:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760733.271191:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.271200:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e8c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.271218:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e8c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.271226:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e8c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20295680->20299775] (req 20295680->20299775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.271386:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d45c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508314240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59881us (60173us total) trans 4294972275 rc 0/0 00000100:00100000:1.0:1789760733.271396:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4987 00000100:00100000:1.0:1789760733.271406:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712269603614 time 712316189486 state 0x2 ref 0 00000100:00100000:3.0:1789760733.276938:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760733.277038:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508314368 00000100:00100000:1.0:1789760733.277050:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508314368 00000100:00100000:1.0:1789760733.277103:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712269603614 time 712316189486 state 0x0 ref 1 00000100:00100000:1.0:1789760733.277118:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712316449902 time 712316189486 state 0x1 ref 1 00000100:00100000:1.0:1789760733.277156:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712316449902 time 712381866191 state 0x0 ref 1 00000100:00100000:1.0:1789760733.277170:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508314368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760733.277219:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20299776, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:1.0:1789760733.277230:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.277246:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760733.277251:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.277257:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.277273:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760733.320705:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20299776 -> 20303872 00010000:00010000:0.0:1789760733.320726:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.320740:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.320748:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.320755:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760733.320760:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.320769:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.320795:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e93 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.320802:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54e93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20299776->20303871] (req 20299776->20303871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.320976:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508314368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43803us (44043us total) trans 4294972276 rc 0/0 00000100:00100000:0.0:1789760733.320987:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4988 00000100:00100000:0.0:1789760733.320996:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712316449902 time 712381866191 state 0x2 ref 0 00000001:00080000:0.0:1789760733.321042:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972276 is committed 00000100:00100000:2.0:1789760733.326834:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760733.326916:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508314496 00000100:00100000:0.0:1789760733.326926:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508314496 00000100:00100000:0.0:1789760733.326980:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712316449902 time 712381866191 state 0x0 ref 1 00000100:00100000:0.0:1789760733.326995:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712382126607 time 712381866191 state 0x1 ref 1 00000100:00100000:0.0:1789760733.327030:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712382126607 time 712431740761 state 0x0 ref 1 00000100:00100000:0.0:1789760733.327043:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508314496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760733.327263:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20303872, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:0.0:1789760733.327272:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e9a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.327335:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760733.327340:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.327346:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.327360:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.378158:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972277 is committed 00002000:00010000:0.0:1789760733.378269:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20303872 -> 20307968 00010000:00010000:0.0:1789760733.378283:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.378314:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.378323:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.378330:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760733.378334:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.378343:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.378430:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.378440:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54e9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20303872->20307967] (req 20303872->20307967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.378640:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508314496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51592us (51810us total) trans 4294972277 rc 0/0 00000100:00100000:0.0:1789760733.378650:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4989 00000100:00100000:0.0:1789760733.378660:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712382126607 time 712431740761 state 0x2 ref 0 00000100:00100000:3.0:1789760733.381387:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760733.381458:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508314624 00000100:00100000:1.0:1789760733.381467:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508314624 00000100:00100000:1.0:1789760733.381529:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712382126607 time 712431740761 state 0x0 ref 1 00000100:00100000:1.0:1789760733.381544:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712432001177 time 712431740761 state 0x1 ref 1 00000100:00100000:1.0:1789760733.381570:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712432001177 time 712486281611 state 0x0 ref 1 00000100:00100000:1.0:1789760733.381582:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508314624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760733.381628:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20307968, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:1.0:1789760733.381637:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54ea1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.381652:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760733.381656:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.381662:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54ea1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.381676:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54ea1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760733.413950:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972278 is committed 00002000:00010000:2.0:1789760733.414035:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20307968 -> 20312064 00010000:00010000:2.0:1789760733.414050:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54ea1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.414063:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54ea1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.414070:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54ea1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.414077:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760733.414117:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54ea1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.414127:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54ea1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.414155:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54ea1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.414162:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe54ea1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20307968->20312063] (req 20307968->20312063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760733.414251:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508314624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32666us (32868us total) trans 4294972278 rc 0/0 00000100:00100000:2.0:1789760733.414262:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4990 00000100:00100000:2.0:1789760733.414271:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712432001177 time 712486281611 state 0x2 ref 0 00000100:00100000:0.0:1789760733.417733:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760733.418220:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508314752 00000100:00100000:3.0:1789760733.418229:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508314752 00000100:00100000:3.0:1789760733.418278:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712432001177 time 712486281611 state 0x0 ref 1 00000100:00100000:3.0:1789760733.418318:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712486542027 time 712486281611 state 0x1 ref 1 00000100:00100000:3.0:1789760733.418340:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712486542027 time 712523050929 state 0x0 ref 1 00000100:00100000:3.0:1789760733.418351:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508314752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760733.418548:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20312064, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:3.0:1789760733.418557:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54ea8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.418571:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760733.418575:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.418581:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54ea8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.418594:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54ea8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.446343:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972279 is committed 00002000:00010000:3.0:1789760733.446515:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20312064 -> 20316160 00010000:00010000:3.0:1789760733.446530:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54ea8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.446544:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54ea8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.446552:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54ea8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.446558:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760733.446563:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54ea8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.446573:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54ea8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.446593:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54ea8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.446601:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54ea8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20312064->20316159] (req 20312064->20316159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760733.446756:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508314752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28401us (29027us total) trans 4294972279 rc 0/0 00000100:00100000:3.0:1789760733.446767:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4991 00000100:00100000:3.0:1789760733.446777:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712486542027 time 712523050929 state 0x2 ref 0 00000100:00100000:3.0:1789760733.449049:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760733.449154:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508314880 00000100:00100000:0.0:1789760733.449163:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508314880 00000100:00100000:0.0:1789760733.449221:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712486542027 time 712523050929 state 0x0 ref 1 00000100:00100000:0.0:1789760733.449236:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712523311345 time 712523050929 state 0x1 ref 1 00000100:00100000:0.0:1789760733.449270:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712523311345 time 712553980089 state 0x0 ref 1 00000100:00100000:0.0:1789760733.449283:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508314880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760733.449363:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20316160, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:0.0:1789760733.449371:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54eaf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.449386:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760733.449391:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.449397:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54eaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.449413:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54eaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760733.476476:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20316160 -> 20320256 00010000:00010000:0.0:1789760733.476505:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54eaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.476519:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54eaf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.476528:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54eaf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.476536:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760733.476541:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54eaf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.476550:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54eaf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.476570:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54eaf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.476578:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54eaf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20316160->20320255] (req 20316160->20320255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.476672:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508314880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27386us (27628us total) trans 4294972280 rc 0/0 00000100:00100000:0.0:1789760733.476683:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4992 00000100:00100000:0.0:1789760733.476693:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712523311345 time 712553980089 state 0x2 ref 0 00000001:00080000:2.0:1789760733.478445:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972280 is committed 00000100:00100000:2.0:1789760733.481440:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760733.481569:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508315008 00000100:00100000:0.0:1789760733.481578:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508315008 00000100:00100000:0.0:1789760733.481632:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712523311345 time 712553980089 state 0x0 ref 1 00000100:00100000:0.0:1789760733.481647:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712554240505 time 712553980089 state 0x1 ref 1 00000100:00100000:0.0:1789760733.481676:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712554240505 time 712586386606 state 0x0 ref 1 00000100:00100000:0.0:1789760733.481687:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508315008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760733.481742:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20320256, mtime: 0x6aad94dd, atime: 0x0, ctime: 0x6aad94dd, blocks: 0xa007 00010000:00010000:0.0:1789760733.481751:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54eb6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.481766:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760733.481770:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.481776:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54eb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.481790:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54eb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.514272:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972281 is committed 00002000:00010000:0.0:1789760733.515614:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20320256 -> 20324352 00002000:00010000:0.0:1789760733.515621:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760733 -> 1789760734 00002000:00010000:0.0:1789760733.515624:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760733 -> 1789760734 00010000:00010000:0.0:1789760733.515637:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54eb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.515649:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54eb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.515658:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54eb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.515665:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760733.515670:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54eb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.515679:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54eb6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.515698:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54eb6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.515705:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54eb6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20320256->20324351] (req 20320256->20324351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.515845:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508315008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34154us (34407us total) trans 4294972281 rc 0/0 00000100:00100000:0.0:1789760733.515855:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4993 00000100:00100000:0.0:1789760733.515864:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712554240505 time 712586386606 state 0x2 ref 0 00000100:00100000:1.0:1789760733.521764:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760733.521861:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508315136 00000100:00100000:3.0:1789760733.521871:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508315136 00000100:00100000:3.0:1789760733.521925:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712554240505 time 712586386606 state 0x0 ref 1 00000100:00100000:3.0:1789760733.521940:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712586647022 time 712586386606 state 0x1 ref 1 00000100:00100000:3.0:1789760733.521968:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712586647022 time 712626679349 state 0x0 ref 1 00000100:00100000:3.0:1789760733.521980:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508315136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760733.522030:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20324352, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:3.0:1789760733.522039:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54ebd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.522054:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760733.522058:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760733.522064:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54ebd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760733.522077:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54ebd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760733.578494:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972282 is committed 00002000:00010000:1.0:1789760733.578728:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20324352 -> 20328448 00010000:00010000:1.0:1789760733.578746:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54ebd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.578759:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54ebd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.578768:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54ebd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.578781:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760733.578786:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54ebd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.578793:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54ebd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.578809:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54ebd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.578813:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe54ebd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20324352->20328447] (req 20324352->20328447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.578894:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508315136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56910us (57133us total) trans 4294972282 rc 0/0 00000100:00100000:1.0:1789760733.578903:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4994 00000100:00100000:1.0:1789760733.578912:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712586647022 time 712626679349 state 0x2 ref 0 00000100:00100000:3.0:1789760733.586161:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760733.586814:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508315264 00000100:00100000:0.0:1789760733.586825:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508315264 00000100:00100000:0.0:1789760733.586882:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712586647022 time 712626679349 state 0x0 ref 1 00000100:00100000:0.0:1789760733.586957:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712626939765 time 712626679349 state 0x1 ref 1 00000100:00100000:0.0:1789760733.586990:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712626939765 time 712691700727 state 0x0 ref 1 00000100:00100000:0.0:1789760733.587003:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508315264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760733.587052:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20328448, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:0.0:1789760733.587061:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54ec4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.587075:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760733.587080:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.587085:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54ec4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.587098:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54ec4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760733.639916:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20328448 -> 20332544 00010000:00010000:1.0:1789760733.639936:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54ec4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.639956:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54ec4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.639971:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54ec4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.639978:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00000001:00080000:3.0:1789760733.639979:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972283 is committed 00010000:00010000:1.0:1789760733.639983:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54ec4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.639991:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54ec4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.640010:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54ec4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.640018:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54ec4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20328448->20332543] (req 20328448->20332543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.640111:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508315264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53105us (53984us total) trans 4294972283 rc 0/0 00000100:00100000:1.0:1789760733.640195:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4995 00000100:00100000:1.0:1789760733.640206:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712626939765 time 712691700727 state 0x2 ref 0 00000100:00100000:2.0:1789760733.646523:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760733.646733:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508315392 00000100:00100000:0.0:1789760733.646743:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508315392 00000100:00100000:0.0:1789760733.646801:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712626939765 time 712691700727 state 0x0 ref 1 00000100:00100000:0.0:1789760733.646817:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712691961143 time 712691700727 state 0x1 ref 1 00000100:00100000:0.0:1789760733.646978:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712691961143 time 712751687622 state 0x0 ref 1 00000100:00100000:0.0:1789760733.646991:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508315392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760733.647038:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20332544, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:0.0:1789760733.647046:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ecb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.647071:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760733.647076:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.647081:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ecb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.647094:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ecb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760733.727685:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972284 is committed 00002000:00010000:0.0:1789760733.727794:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20332544 -> 20336640 00010000:00010000:0.0:1789760733.727808:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ecb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.727826:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ecb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.727835:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ecb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.727841:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760733.727846:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ecb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.727855:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ecb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.727874:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ecb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.727881:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ecb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20332544->20336639] (req 20332544->20336639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760733.728245:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508315392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 81251us (81726us total) trans 4294972284 rc 0/0 00000100:00100000:0.0:1789760733.728255:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4996 00000100:00100000:0.0:1789760733.728264:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712691961143 time 712751687622 state 0x2 ref 0 00000100:00100000:0.0:1789760733.737282:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760733.737406:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508315520 00000100:00100000:1.0:1789760733.737416:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508315520 00000100:00100000:1.0:1789760733.737470:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712691961143 time 712751687622 state 0x0 ref 1 00000100:00100000:1.0:1789760733.737488:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712751948038 time 712751687622 state 0x1 ref 1 00000100:00100000:1.0:1789760733.737517:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712751948038 time 712842227264 state 0x0 ref 1 00000100:00100000:1.0:1789760733.737529:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508315520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760733.737578:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20336640, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:1.0:1789760733.737587:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ed2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.737602:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760733.737606:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.737613:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ed2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.737627:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ed2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760733.810566:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972285 is committed 00002000:00010000:1.0:1789760733.810661:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20336640 -> 20340736 00010000:00010000:1.0:1789760733.810676:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ed2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.810688:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ed2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.810696:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ed2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.810707:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760733.810711:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ed2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.810719:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ed2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.810736:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ed2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.810743:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ed2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20336640->20340735] (req 20336640->20340735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.810826:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508315520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 73294us (73550us total) trans 4294972285 rc 0/0 00000100:00100000:1.0:1789760733.810835:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4997 00000100:00100000:1.0:1789760733.810842:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712751948038 time 712842227264 state 0x2 ref 0 00000100:00100000:3.0:1789760733.813554:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760733.813631:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508315648 00000100:00100000:1.0:1789760733.813641:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508315648 00000100:00100000:1.0:1789760733.813694:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712751948038 time 712842227264 state 0x0 ref 1 00000100:00100000:1.0:1789760733.813709:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712842487680 time 712842227264 state 0x1 ref 1 00000100:00100000:1.0:1789760733.813739:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712842487680 time 712918449276 state 0x0 ref 1 00000100:00100000:1.0:1789760733.813754:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508315648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760733.813801:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20340736, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:1.0:1789760733.813810:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ed9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.813825:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:1.0:1789760733.813830:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.813835:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ed9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.813848:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ed9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.852718:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972286 is committed 00002000:00010000:1.0:1789760733.852861:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20340736 -> 20344832 00010000:00010000:1.0:1789760733.852877:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ed9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.852890:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ed9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.852899:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ed9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.852907:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760733.852912:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ed9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.852922:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ed9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.852943:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ed9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.852951:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ed9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20340736->20344831] (req 20340736->20344831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.853048:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508315648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39291us (39497us total) trans 4294972286 rc 0/0 00000100:00100000:1.0:1789760733.853066:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4998 00000100:00100000:1.0:1789760733.853075:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712842487680 time 712918449276 state 0x2 ref 0 00000100:00100000:3.0:1789760733.861018:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760733.861230:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508315776 00000100:00100000:0.0:1789760733.861240:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508315776 00000100:00100000:0.0:1789760733.861391:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712842487680 time 712918449276 state 0x0 ref 1 00000100:00100000:0.0:1789760733.861408:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712918709692 time 712918449276 state 0x1 ref 1 00000100:00100000:0.0:1789760733.861492:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712918709692 time 712966201468 state 0x0 ref 1 00000100:00100000:0.0:1789760733.861514:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508315776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760733.861561:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20344832, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:0.0:1789760733.861570:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ee0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.861586:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760733.861590:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760733.861597:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ee0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760733.861610:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ee0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.896925:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972287 is committed 00002000:00010000:1.0:1789760733.897025:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20344832 -> 20348928 00010000:00010000:1.0:1789760733.897039:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ee0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.897116:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ee0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.897125:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ee0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.897141:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760733.897146:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ee0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.897156:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ee0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.897177:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ee0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.897185:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54ee0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20344832->20348927] (req 20344832->20348927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.897279:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508315776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35771us (36264us total) trans 4294972287 rc 0/0 00000100:00100000:1.0:1789760733.897321:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 4999 00000100:00100000:1.0:1789760733.897330:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712918709692 time 712966201468 state 0x2 ref 0 00000100:00100000:2.0:1789760733.902802:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760733.902886:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508315904 00000100:00100000:1.0:1789760733.902895:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508315904 00000100:00100000:1.0:1789760733.902946:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712918709692 time 712966201468 state 0x0 ref 1 00000100:00100000:1.0:1789760733.902961:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712966461884 time 712966201468 state 0x1 ref 1 00000100:00100000:1.0:1789760733.902992:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712966461884 time 713007701931 state 0x0 ref 1 00000100:00100000:1.0:1789760733.903004:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508315904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760733.903059:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20348928, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:1.0:1789760733.903079:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ee7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.903094:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760733.903099:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.903105:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ee7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.903118:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ee7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760733.938341:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972288 is committed 00002000:00010000:1.0:1789760733.938775:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20348928 -> 20353024 00010000:00010000:1.0:1789760733.938791:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ee7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.938874:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ee7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.938884:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ee7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.938891:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760733.938896:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ee7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.938906:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ee7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.938929:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ee7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.938936:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54ee7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20348928->20353023] (req 20348928->20353023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.939073:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508315904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36058us (36269us total) trans 4294972288 rc 0/0 00000100:00100000:1.0:1789760733.939083:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5000 00000100:00100000:1.0:1789760733.939093:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712966461884 time 713007701931 state 0x2 ref 0 00000100:00100000:3.0:1789760733.942794:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760733.942865:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508316032 00000100:00100000:1.0:1789760733.942879:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508316032 00000100:00100000:1.0:1789760733.942937:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 712966461884 time 713007701931 state 0x0 ref 1 00000100:00100000:1.0:1789760733.942952:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713007962347 time 713007701931 state 0x1 ref 1 00000100:00100000:1.0:1789760733.943032:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713007962347 time 713047741980 state 0x0 ref 1 00000100:00100000:1.0:1789760733.943045:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508316032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760733.943094:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20353024, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:1.0:1789760733.943103:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54eee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.943119:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760733.943124:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.943140:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54eee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.943155:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54eee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760733.950833:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760733.973350:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972289 is committed 00002000:00010000:1.0:1789760733.973455:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20353024 -> 20357120 00010000:00010000:1.0:1789760733.973470:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54eee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.973497:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54eee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.973506:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54eee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.973513:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760733.973517:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54eee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.973526:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54eee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.973545:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54eee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.973552:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54eee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20353024->20357119] (req 20353024->20357119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760733.973644:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508316032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30596us (30861us total) trans 4294972289 rc 0/0 00000100:00100000:1.0:1789760733.973654:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5001 00000100:00100000:1.0:1789760733.973665:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713007962347 time 713047741980 state 0x2 ref 0 00000100:00100000:3.0:1789760733.980001:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760733.980084:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508316160 00000100:00100000:1.0:1789760733.980093:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508316160 00000100:00100000:1.0:1789760733.980155:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713007962347 time 713047741980 state 0x0 ref 1 00000100:00100000:1.0:1789760733.980171:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713048002396 time 713047741980 state 0x1 ref 1 00000100:00100000:1.0:1789760733.980199:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713048002396 time 713084910120 state 0x0 ref 1 00000100:00100000:1.0:1789760733.980221:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508316160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760733.980266:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20357120, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:1.0:1789760733.980275:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ef5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.980320:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760733.980325:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760733.980336:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ef5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760733.980351:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ef5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.004250:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972290 is committed 00002000:00010000:1.0:1789760734.004358:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20357120 -> 20361216 00010000:00010000:1.0:1789760734.004374:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ef5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.004386:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ef5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.004395:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ef5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.004402:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760734.004406:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ef5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.004414:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ef5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.004434:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ef5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.004441:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54ef5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20357120->20361215] (req 20357120->20361215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.004648:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508316160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24432us (24648us total) trans 4294972290 rc 0/0 00000100:00100000:1.0:1789760734.004663:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5002 00000100:00100000:1.0:1789760734.004673:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713048002396 time 713084910120 state 0x2 ref 0 00000100:00100000:2.0:1789760734.013453:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760734.013645:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508316288 00000100:00100000:1.0:1789760734.013655:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508316288 00000100:00100000:1.0:1789760734.013713:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713048002396 time 713084910120 state 0x0 ref 1 00000100:00100000:1.0:1789760734.013728:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713085170536 time 713084910120 state 0x1 ref 1 00000100:00100000:1.0:1789760734.013789:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713085170536 time 713118499607 state 0x0 ref 1 00000100:00100000:1.0:1789760734.013810:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508316288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760734.013860:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20361216, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:1.0:1789760734.013869:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54efc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.013885:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760734.013889:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.013894:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54efc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.013907:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54efc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.062523:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972291 is committed 00002000:00010000:1.0:1789760734.062687:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20361216 -> 20365312 00010000:00010000:1.0:1789760734.062702:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54efc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.062716:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54efc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.062726:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54efc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.062733:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760734.062738:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54efc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.062747:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54efc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.062768:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54efc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.062777:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54efc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20361216->20365311] (req 20361216->20365311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.062945:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508316288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49130us (49497us total) trans 4294972291 rc 0/0 00000100:00100000:1.0:1789760734.062956:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5003 00000100:00100000:1.0:1789760734.062966:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713085170536 time 713118499607 state 0x2 ref 0 00000100:00100000:3.0:1789760734.066875:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760734.067079:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508316416 00000100:00100000:1.0:1789760734.067089:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508316416 00000100:00100000:1.0:1789760734.067150:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713085170536 time 713118499607 state 0x0 ref 1 00000100:00100000:1.0:1789760734.067164:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713118760023 time 713118499607 state 0x1 ref 1 00000100:00100000:1.0:1789760734.067236:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713118760023 time 713171946424 state 0x0 ref 1 00000100:00100000:1.0:1789760734.067249:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508316416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760734.067345:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20365312, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:1.0:1789760734.067354:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.067370:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760734.067375:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.067381:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.067394:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.079944:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04985469180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534066432:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1789760734.080070:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760734.080320:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534066432 00000100:00100000:0.0:1789760734.080329:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534066432 00000100:00100000:0.0:1789760734.080374:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:0.0:1789760734.080383:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534066432:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760734.080424:0:7358:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789760734.082739:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534066432:12345-0@lo:41:osp-pre-1-0.0 Request processed in 2346us (2658us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760734.082754:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00100000:3.0:1789760734.082978:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04985469180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534066432:0@lo:41:osp-pre-1-0.0 00000001:00080000:3.0:1789760734.101092:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972292 is committed 00002000:00010000:1.0:1789760734.101179:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20365312 -> 20369408 00010000:00010000:1.0:1789760734.101191:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.101200:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.101210:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.101214:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760734.101218:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.101223:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.101238:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f03 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.101242:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20365312->20369407] (req 20365312->20369407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.101569:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508316416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34316us (34700us total) trans 4294972292 rc 0/0 00000100:00100000:1.0:1789760734.101580:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5004 00000100:00100000:1.0:1789760734.101589:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713118760023 time 713171946424 state 0x2 ref 0 00000100:00100000:3.0:1789760734.107759:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760734.107854:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508316544 00000100:00100000:1.0:1789760734.107863:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508316544 00000100:00100000:1.0:1789760734.107915:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713118760023 time 713171946424 state 0x0 ref 1 00000100:00100000:1.0:1789760734.107931:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713172206840 time 713171946424 state 0x1 ref 1 00000100:00100000:1.0:1789760734.107966:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713172206840 time 713212676327 state 0x0 ref 1 00000100:00100000:1.0:1789760734.107979:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508316544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760734.108025:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20369408, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:1.0:1789760734.108035:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.108051:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760734.108056:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.108063:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.108078:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760734.158688:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20369408 -> 20373504 00010000:00010000:1.0:1789760734.158707:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.158719:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.158728:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.158735:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760734.158740:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.158749:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.158771:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.158779:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20369408->20373503] (req 20369408->20373503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760734.158842:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972293 is committed 00000100:00100000:1.0:1789760734.158909:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508316544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50925us (51154us total) trans 4294972293 rc 0/0 00000100:00100000:1.0:1789760734.158921:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5005 00000100:00100000:1.0:1789760734.158931:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713172206840 time 713212676327 state 0x2 ref 0 00000100:00100000:2.0:1789760734.162985:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760734.163913:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508316672 00000100:00100000:1.0:1789760734.163924:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508316672 00000100:00100000:1.0:1789760734.163983:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713172206840 time 713212676327 state 0x0 ref 1 00000100:00100000:1.0:1789760734.163999:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713212936743 time 713212676327 state 0x1 ref 1 00000100:00100000:1.0:1789760734.164035:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713212936743 time 713268745990 state 0x0 ref 1 00000100:00100000:1.0:1789760734.164047:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508316672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760734.164676:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20373504, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:1.0:1789760734.164687:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54f11 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.164702:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760734.164706:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.164713:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54f11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.164727:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54f11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.204994:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972294 is committed 00002000:00010000:3.0:1789760734.205691:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20373504 -> 20377600 00010000:00010000:3.0:1789760734.205706:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54f11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.205718:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54f11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.205726:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54f11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.205733:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760734.205738:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54f11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.205746:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54f11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.205769:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54f11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.205777:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe54f11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20373504->20377599] (req 20373504->20377599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.205921:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508316672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41870us (42939us total) trans 4294972294 rc 0/0 00000100:00100000:3.0:1789760734.205933:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5006 00000100:00100000:3.0:1789760734.205944:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713212936743 time 713268745990 state 0x2 ref 0 00000100:00100000:0.0:1789760734.217990:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760734.218180:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508316800 00000100:00100000:1.0:1789760734.218189:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508316800 00000100:00100000:1.0:1789760734.218242:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713212936743 time 713268745990 state 0x0 ref 1 00000100:00100000:1.0:1789760734.218257:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713269006406 time 713268745990 state 0x1 ref 1 00000100:00100000:1.0:1789760734.218361:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713269006406 time 713323070039 state 0x0 ref 1 00000100:00100000:1.0:1789760734.218375:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508316800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760734.218421:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20377600, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:1.0:1789760734.218430:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.218446:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760734.218449:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.218456:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.218468:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.267417:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972295 is committed 00002000:00010000:1.0:1789760734.267567:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20377600 -> 20381696 00010000:00010000:1.0:1789760734.267581:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.267712:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.267721:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.267728:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760734.267732:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.267741:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f18 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.267760:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f18 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.267766:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20377600->20381695] (req 20377600->20381695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.267906:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508316800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49528us (49919us total) trans 4294972295 rc 0/0 00000100:00100000:1.0:1789760734.267916:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5007 00000100:00100000:1.0:1789760734.267925:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713269006406 time 713323070039 state 0x2 ref 0 00000100:00100000:0.0:1789760734.272335:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760734.272442:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508316928 00000100:00100000:1.0:1789760734.272451:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508316928 00000100:00100000:1.0:1789760734.272516:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713269006406 time 713323070039 state 0x0 ref 1 00000100:00100000:1.0:1789760734.272530:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713323330455 time 713323070039 state 0x1 ref 1 00000100:00100000:1.0:1789760734.272565:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713323330455 time 713377275183 state 0x0 ref 1 00000100:00100000:1.0:1789760734.272577:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508316928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760734.272643:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20381696, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:1.0:1789760734.272651:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54f1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.272666:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760734.272670:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.272676:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54f1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.272689:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54f1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.306970:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972296 is committed 00002000:00010000:2.0:1789760734.307253:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20381696 -> 20385792 00010000:00010000:2.0:1789760734.307268:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54f1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.307279:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54f1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.307303:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54f1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.307312:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760734.307316:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54f1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.307325:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54f1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.307343:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54f1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.307350:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe54f1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20381696->20385791] (req 20381696->20385791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.307441:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508316928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34860us (35110us total) trans 4294972296 rc 0/0 00000100:00100000:2.0:1789760734.307452:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5008 00000100:00100000:2.0:1789760734.307461:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713323330455 time 713377275183 state 0x2 ref 0 00000100:00100000:2.0:1789760734.311854:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760734.315778:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508317056 00000100:00100000:3.0:1789760734.315794:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508317056 00000100:00100000:3.0:1789760734.315854:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713323330455 time 713377275183 state 0x0 ref 1 00000100:00100000:3.0:1789760734.315867:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713377535599 time 713377275183 state 0x1 ref 1 00000100:00100000:3.0:1789760734.315933:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713377535599 time 713420643826 state 0x0 ref 1 00000100:00100000:3.0:1789760734.315942:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508317056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760734.315982:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20385792, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:3.0:1789760734.315989:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54f26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.316001:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760734.316004:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.316008:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54f26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.316018:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54f26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.383563:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972297 is committed 00002000:00010000:3.0:1789760734.383614:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20385792 -> 20389888 00010000:00010000:3.0:1789760734.383630:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54f26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.383642:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54f26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.383651:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54f26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.383659:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760734.383664:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54f26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.383673:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54f26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.383692:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54f26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.383705:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54f26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20385792->20389887] (req 20385792->20389887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.383910:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508317056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 67964us (72061us total) trans 4294972297 rc 0/0 00000100:00100000:3.0:1789760734.383920:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5009 00000100:00100000:3.0:1789760734.383929:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713377535599 time 713420643826 state 0x2 ref 0 00000100:00100000:0.0:1789760734.391864:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760734.391949:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508317184 00000100:00100000:1.0:1789760734.391959:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508317184 00000100:00100000:1.0:1789760734.392012:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713377535599 time 713420643826 state 0x0 ref 1 00000100:00100000:1.0:1789760734.392028:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713420904242 time 713420643826 state 0x1 ref 1 00000100:00100000:1.0:1789760734.392059:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713420904242 time 713496769457 state 0x0 ref 1 00000100:00100000:1.0:1789760734.392071:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508317184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760734.392137:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20389888, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:1.0:1789760734.392147:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.392163:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760734.392167:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.392174:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.392187:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760734.478917:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20389888 -> 20393984 00010000:00010000:1.0:1789760734.478937:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.478956:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.478965:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.478972:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760734.478977:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.478987:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.479007:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.479474:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe54f2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20389888->20393983] (req 20389888->20393983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.479595:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508317184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 87522us (87738us total) trans 4294972298 rc 0/0 00000100:00100000:1.0:1789760734.479607:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5010 00000100:00100000:1.0:1789760734.479616:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713420904242 time 713496769457 state 0x2 ref 0 00000001:00080000:1.0:1789760734.479658:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972298 is committed 00000100:00100000:0.0:1789760734.483403:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760734.483465:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508317312 00000100:00100000:2.0:1789760734.483474:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508317312 00000100:00100000:2.0:1789760734.483528:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713420904242 time 713496769457 state 0x0 ref 1 00000100:00100000:2.0:1789760734.483545:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713497029873 time 713496769457 state 0x1 ref 1 00000100:00100000:2.0:1789760734.483615:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713497029873 time 713588323360 state 0x0 ref 1 00000100:00100000:2.0:1789760734.483629:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508317312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760734.483676:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20393984, mtime: 0x6aad94de, atime: 0x0, ctime: 0x6aad94de, blocks: 0xa007 00010000:00010000:2.0:1789760734.483685:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54f34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.483699:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760734.483703:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.483709:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54f34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.483723:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54f34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760734.547826:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972299 is committed 00002000:00010000:0.0:1789760734.547936:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20393984 -> 20398080 00002000:00010000:0.0:1789760734.547941:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760734 -> 1789760735 00002000:00010000:0.0:1789760734.547944:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760734 -> 1789760735 00010000:00010000:0.0:1789760734.547955:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54f34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.547968:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54f34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.547978:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54f34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.547985:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760734.547989:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54f34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.547998:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54f34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.548017:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54f34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.548025:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe54f34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20393984->20398079] (req 20393984->20398079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.548136:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508317312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 64496us (64728us total) trans 4294972299 rc 0/0 00000100:00100000:0.0:1789760734.548146:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5011 00000100:00100000:0.0:1789760734.548155:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713497029873 time 713588323360 state 0x2 ref 0 00000100:00100000:2.0:1789760734.555056:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760734.555177:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508317440 00000100:00100000:3.0:1789760734.555188:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508317440 00000100:00100000:3.0:1789760734.555242:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713497029873 time 713588323360 state 0x0 ref 1 00000100:00100000:3.0:1789760734.555258:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713588583776 time 713588323360 state 0x1 ref 1 00000100:00100000:3.0:1789760734.555320:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713588583776 time 713660029378 state 0x0 ref 1 00000100:00100000:3.0:1789760734.555333:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508317440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760734.555381:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20398080, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:3.0:1789760734.555390:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54f3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.555405:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760734.555409:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760734.555416:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54f3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.555437:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54f3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760734.609748:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20398080 -> 20402176 00010000:00010000:3.0:1789760734.609769:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54f3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.609783:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54f3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.609792:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54f3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.609800:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760734.609805:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54f3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.609815:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54f3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.609835:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54f3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.609844:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54f3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20398080->20402175] (req 20398080->20402175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.610158:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508317440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54820us (55114us total) trans 4294972300 rc 0/0 00000100:00100000:3.0:1789760734.610169:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5012 00000100:00100000:3.0:1789760734.610179:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713588583776 time 713660029378 state 0x2 ref 0 00000001:00080000:3.0:1789760734.610217:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972300 is committed 00000100:00100000:3.0:1789760734.615925:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760734.617405:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508317568 00000100:00100000:1.0:1789760734.617417:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508317568 00000100:00100000:1.0:1789760734.617563:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713588583776 time 713660029378 state 0x0 ref 1 00000100:00100000:1.0:1789760734.617590:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713660289794 time 713660029378 state 0x1 ref 1 00000100:00100000:1.0:1789760734.617622:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713660289794 time 713722332741 state 0x0 ref 1 00000100:00100000:1.0:1789760734.617637:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508317568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760734.617679:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20402176, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:1.0:1789760734.617687:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54f42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.617701:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760734.617705:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.617710:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54f42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.617723:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54f42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.662037:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972301 is committed 00002000:00010000:3.0:1789760734.666272:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20402176 -> 20406272 00010000:00010000:3.0:1789760734.666334:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54f42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.666348:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54f42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.666357:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54f42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.666364:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760734.666369:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54f42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.666377:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54f42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.666396:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54f42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760734.666404:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe54f42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20402176->20406271] (req 20402176->20406271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760734.666588:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508317568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48948us (50668us total) trans 4294972301 rc 0/0 00000100:00100000:3.0:1789760734.666600:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5013 00000100:00100000:3.0:1789760734.666609:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713660289794 time 713722332741 state 0x2 ref 0 00000100:00100000:1.0:1789760734.678699:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760734.678806:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508317696 00000100:00100000:0.0:1789760734.678816:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508317696 00000100:00100000:0.0:1789760734.678874:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713660289794 time 713722332741 state 0x0 ref 1 00000100:00100000:0.0:1789760734.678889:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713722593157 time 713722332741 state 0x1 ref 1 00000100:00100000:0.0:1789760734.678924:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713722593157 time 713783634587 state 0x0 ref 1 00000100:00100000:0.0:1789760734.678936:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508317696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760734.678984:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20406272, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:0.0:1789760734.678993:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54f49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.679010:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760734.679015:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.679021:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54f49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.679040:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54f49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760734.770326:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20406272 -> 20410368 00010000:00010000:1.0:1789760734.770346:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54f49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.770365:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54f49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.770374:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54f49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.770382:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760734.770386:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54f49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.770395:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54f49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.770414:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54f49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.770421:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe54f49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20406272->20410367] (req 20406272->20410367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.770524:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508317696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 91584us (91829us total) trans 4294972302 rc 0/0 00000100:00100000:1.0:1789760734.770535:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5014 00000100:00100000:1.0:1789760734.770544:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713722593157 time 713783634587 state 0x2 ref 0 00000001:00080000:1.0:1789760734.770582:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972302 is committed 00000100:00100000:2.0:1789760734.777206:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760734.777810:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508317824 00000100:00100000:3.0:1789760734.777820:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508317824 00000100:00100000:2.0:1789760734.777959:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713722593157 time 713783634587 state 0x0 ref 1 00000100:00100000:2.0:1789760734.777976:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713783895003 time 713783634587 state 0x1 ref 1 00000100:00100000:2.0:1789760734.778049:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713783895003 time 713882758588 state 0x0 ref 1 00000100:00100000:2.0:1789760734.778063:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508317824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760734.778112:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20410368, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:2.0:1789760734.778121:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54f50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.778147:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760734.778152:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760734.778157:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54f50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.778172:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54f50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760734.819835:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972303 is committed 00002000:00010000:0.0:1789760734.819920:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20410368 -> 20414464 00010000:00010000:0.0:1789760734.819935:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54f50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.819948:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54f50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.819957:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54f50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.819965:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760734.819970:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54f50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.819980:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54f50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.820000:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54f50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.820008:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54f50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20410368->20414463] (req 20410368->20414463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760734.820202:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508317824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42136us (43001us total) trans 4294972303 rc 0/0 00000100:00100000:0.0:1789760734.820213:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5015 00000100:00100000:0.0:1789760734.820223:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713783895003 time 713882758588 state 0x2 ref 0 00000100:00100000:3.0:1789760734.824200:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760734.824323:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508317952 00000100:00100000:1.0:1789760734.824333:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508317952 00000100:00100000:1.0:1789760734.824384:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713783895003 time 713882758588 state 0x0 ref 1 00000100:00100000:1.0:1789760734.824400:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713883019004 time 713882758588 state 0x1 ref 1 00000100:00100000:1.0:1789760734.824588:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713883019004 time 713929297220 state 0x0 ref 1 00000100:00100000:1.0:1789760734.824601:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508317952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760734.824652:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20414464, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:1.0:1789760734.824661:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.824677:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760734.824682:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.824689:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.824703:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760734.859425:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20414464 -> 20418560 00010000:00010000:1.0:1789760734.859444:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.859457:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.859468:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.859486:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760734.859492:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.859501:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.859521:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.859529:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20414464->20418559] (req 20414464->20418559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.859738:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508317952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35125us (35533us total) trans 4294972304 rc 0/0 00000100:00100000:1.0:1789760734.859749:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5016 00000100:00100000:1.0:1789760734.859761:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713883019004 time 713929297220 state 0x2 ref 0 00000001:00080000:0.0:1789760734.859892:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972304 is committed 00000100:00100000:2.0:1789760734.868971:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760734.869063:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508318080 00000100:00100000:1.0:1789760734.869072:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508318080 00000100:00100000:1.0:1789760734.869136:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713883019004 time 713929297220 state 0x0 ref 1 00000100:00100000:1.0:1789760734.869152:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713929557636 time 713929297220 state 0x1 ref 1 00000100:00100000:1.0:1789760734.869183:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713929557636 time 713973893520 state 0x0 ref 1 00000100:00100000:1.0:1789760734.869196:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508318080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760734.869249:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20418560, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:1.0:1789760734.869258:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.869275:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=713s v=(0 0) 00010000:00010000:1.0:1789760734.869279:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760734.869322:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.869338:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760734.899997:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972305 is committed 00002000:00010000:1.0:1789760734.900166:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20418560 -> 20422656 00010000:00010000:1.0:1789760734.900180:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.900192:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.900200:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.900207:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760734.900212:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.900221:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.900241:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760734.900248:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20418560->20422655] (req 20418560->20422655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760734.900370:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508318080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31171us (31401us total) trans 4294972305 rc 0/0 00000100:00100000:1.0:1789760734.900380:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5017 00000100:00100000:1.0:1789760734.900389:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713929557636 time 713973893520 state 0x2 ref 0 00000100:00100000:3.0:1789760734.916740:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760734.917183:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508318208 00000100:00100000:0.0:1789760734.917201:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508318208 00000100:00100000:0.0:1789760734.917258:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713929557636 time 713973893520 state 0x0 ref 1 00000100:00100000:0.0:1789760734.917274:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713974153936 time 713973893520 state 0x1 ref 1 00000100:00100000:0.0:1789760734.917337:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713974153936 time 714022047038 state 0x0 ref 1 00000100:00100000:0.0:1789760734.917350:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508318208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760734.917397:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20422656, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:0.0:1789760734.917406:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54f65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.917422:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760734.917426:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.917433:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54f65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.917454:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54f65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.974675:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760734.979488:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972306 is committed 00002000:00010000:2.0:1789760734.979557:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20422656 -> 20426752 00010000:00010000:2.0:1789760734.979582:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54f65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.979595:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54f65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.979603:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54f65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.979611:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760734.979616:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54f65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.979625:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54f65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.979649:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54f65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760734.979657:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe54f65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20422656->20426751] (req 20422656->20426751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760734.979760:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508318208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 62407us (63034us total) trans 4294972306 rc 0/0 00000100:00100000:2.0:1789760734.979770:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5018 00000100:00100000:2.0:1789760734.979780:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713974153936 time 714022047038 state 0x2 ref 0 00000100:00100000:3.0:1789760734.992064:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760734.992180:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508318336 00000100:00100000:1.0:1789760734.992190:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508318336 00000100:00100000:1.0:1789760734.992247:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 713974153936 time 714022047038 state 0x0 ref 1 00000100:00100000:1.0:1789760734.992263:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714022307454 time 714022047038 state 0x1 ref 1 00000100:00100000:1.0:1789760734.992339:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714022307454 time 714097048118 state 0x0 ref 1 00000100:00100000:1.0:1789760734.992353:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508318336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760734.992459:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20426752, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:0.0:1789760734.992470:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f6c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.992486:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760734.992490:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760734.992497:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760734.992512:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.061601:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972307 is committed 00002000:00010000:2.0:1789760735.061876:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20426752 -> 20430848 00010000:00010000:2.0:1789760735.061890:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.061908:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.061918:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.061925:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760735.061930:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.061944:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.061968:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.061976:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe54f6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20426752->20430847] (req 20426752->20430847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.062067:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508318336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 69713us (70009us total) trans 4294972307 rc 0/0 00000100:00100000:2.0:1789760735.062078:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5019 00000100:00100000:2.0:1789760735.062087:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714022307454 time 714097048118 state 0x2 ref 0 00000100:00100000:3.0:1789760735.065938:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760735.066163:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508318464 00000100:00100000:1.0:1789760735.066174:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508318464 00000100:00100000:1.0:1789760735.066240:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714022307454 time 714097048118 state 0x0 ref 1 00000100:00100000:1.0:1789760735.066255:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714097308534 time 714097048118 state 0x1 ref 1 00000100:00100000:1.0:1789760735.066466:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714097308534 time 714170994072 state 0x0 ref 1 00000100:00100000:1.0:1789760735.066487:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508318464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760735.066529:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20430848, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:1.0:1789760735.066543:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54f73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.066559:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760735.066563:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.066569:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54f73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.066582:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54f73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760735.067108:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760735.097825:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972308 is committed 00002000:00010000:3.0:1789760735.097922:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20430848 -> 20434944 00010000:00010000:3.0:1789760735.097938:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54f73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.097951:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54f73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.097960:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54f73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.097972:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760735.097977:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54f73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.097985:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54f73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.098004:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54f73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.098012:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe54f73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20430848->20434943] (req 20430848->20434943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.098109:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508318464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31623us (32177us total) trans 4294972308 rc 0/0 00000100:00100000:3.0:1789760735.098129:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5020 00000100:00100000:3.0:1789760735.098139:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714097308534 time 714170994072 state 0x2 ref 0 00000100:00100000:3.0:1789760735.103239:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760735.103804:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508318592 00000100:00100000:2.0:1789760735.103815:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508318592 00000100:00100000:2.0:1789760735.103870:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714097308534 time 714170994072 state 0x0 ref 1 00000100:00100000:2.0:1789760735.103887:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714171254488 time 714170994072 state 0x1 ref 1 00000100:00100000:2.0:1789760735.103919:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714171254488 time 714208629078 state 0x0 ref 1 00000100:00100000:2.0:1789760735.103931:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508318592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760735.103978:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20434944, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:2.0:1789760735.103987:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54f7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.104004:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.104008:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.104014:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54f7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.104029:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54f7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.132556:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972309 is committed 00002000:00010000:0.0:1789760735.132690:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20434944 -> 20439040 00010000:00010000:0.0:1789760735.132705:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54f7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.132718:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54f7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.132726:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54f7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.132734:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760735.132739:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54f7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.132748:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54f7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.132768:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54f7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.132775:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe54f7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20434944->20439039] (req 20434944->20439039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.132865:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508318592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28931us (29631us total) trans 4294972309 rc 0/0 00000100:00100000:0.0:1789760735.132876:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5021 00000100:00100000:0.0:1789760735.132885:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714171254488 time 714208629078 state 0x2 ref 0 00000100:00100000:0.0:1789760735.136800:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760735.137024:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508318720 00000100:00100000:2.0:1789760735.137043:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508318720 00000100:00100000:2.0:1789760735.137094:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714171254488 time 714208629078 state 0x0 ref 1 00000100:00100000:2.0:1789760735.137108:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714208889494 time 714208629078 state 0x1 ref 1 00000100:00100000:2.0:1789760735.137182:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714208889494 time 714241891582 state 0x0 ref 1 00000100:00100000:2.0:1789760735.137195:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508318720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760735.137241:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20439040, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:2.0:1789760735.137250:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54f81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.137262:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.137265:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.137269:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54f81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.137283:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54f81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760735.175865:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972310 is committed 00002000:00010000:0.0:1789760735.176825:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20439040 -> 20443136 00010000:00010000:0.0:1789760735.176842:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54f81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.176855:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54f81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.176869:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54f81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.176877:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760735.176882:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54f81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.176891:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54f81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.176909:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54f81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.176917:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe54f81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20439040->20443135] (req 20439040->20443135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.177017:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508318720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39820us (40222us total) trans 4294972310 rc 0/0 00000100:00100000:0.0:1789760735.177028:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5022 00000100:00100000:0.0:1789760735.177037:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714208889494 time 714241891582 state 0x2 ref 0 00000100:00100000:3.0:1789760735.183022:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760735.184895:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508318848 00000100:00100000:1.0:1789760735.184909:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508318848 00000100:00100000:1.0:1789760735.184972:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714208889494 time 714241891582 state 0x0 ref 1 00000100:00100000:1.0:1789760735.184986:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714242151998 time 714241891582 state 0x1 ref 1 00000100:00100000:1.0:1789760735.185021:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714242151998 time 714289731598 state 0x0 ref 1 00000100:00100000:1.0:1789760735.185374:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508318848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760735.185423:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20443136, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:1.0:1789760735.185433:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.185447:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760735.185452:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.185463:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.185544:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760735.226724:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20443136 -> 20447232 00010000:00010000:1.0:1789760735.226744:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.226765:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.226775:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.226782:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760735.226787:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.226801:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.226820:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.226828:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20443136->20447231] (req 20443136->20447231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.226921:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508318848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41880us (43904us total) trans 4294972311 rc 0/0 00000100:00100000:1.0:1789760735.226930:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5023 00000100:00100000:1.0:1789760735.226940:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714242151998 time 714289731598 state 0x2 ref 0 00000001:00080000:2.0:1789760735.227278:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972311 is committed 00000100:00100000:2.0:1789760735.231801:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760735.231880:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508318976 00000100:00100000:0.0:1789760735.231890:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508318976 00000100:00100000:0.0:1789760735.231937:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714242151998 time 714289731598 state 0x0 ref 1 00000100:00100000:0.0:1789760735.231952:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714289992014 time 714289731598 state 0x1 ref 1 00000100:00100000:0.0:1789760735.231980:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714289992014 time 714336690320 state 0x0 ref 1 00000100:00100000:0.0:1789760735.231991:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508318976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760735.232046:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20447232, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:0.0:1789760735.232056:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54f8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.232071:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760735.232075:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.232081:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54f8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.232095:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54f8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.284863:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972312 is committed 00002000:00010000:2.0:1789760735.285218:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20447232 -> 20451328 00010000:00010000:2.0:1789760735.285233:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54f8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.285246:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54f8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.285255:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54f8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.285262:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760735.285267:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54f8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.285276:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54f8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.285372:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54f8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.285387:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe54f8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20447232->20451327] (req 20447232->20451327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.285521:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508318976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53526us (53724us total) trans 4294972312 rc 0/0 00000100:00100000:2.0:1789760735.285533:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5024 00000100:00100000:2.0:1789760735.285543:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714289992014 time 714336690320 state 0x2 ref 0 00000100:00100000:1.0:1789760735.301069:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760735.301154:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508319104 00000100:00100000:0.0:1789760735.301165:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508319104 00000100:00100000:0.0:1789760735.301221:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714289992014 time 714336690320 state 0x0 ref 1 00000100:00100000:0.0:1789760735.301237:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714336950736 time 714336690320 state 0x1 ref 1 00000100:00100000:0.0:1789760735.301267:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714336950736 time 714405976748 state 0x0 ref 1 00000100:00100000:0.0:1789760735.301279:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508319104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760735.301354:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20451328, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:0.0:1789760735.301362:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54f96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.301377:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760735.301382:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.301399:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54f96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.301414:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54f96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.306457:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760735.306500:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760735.306537:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760735.306565:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760735.306593:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760735.306640:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508319360 00000100:00100000:1.0:1789760735.306650:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508319360 00000100:00100000:1.0:1789760735.306713:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508319616 00000100:00100000:1.0:1789760735.306717:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508319616 00000100:00100000:1.0:1789760735.306741:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508319488 00000100:00100000:1.0:1789760735.306745:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508319488 00000100:00100000:1.0:1789760735.306764:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508319744 00000100:00100000:1.0:1789760735.306768:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508319744 00000100:00100000:1.0:1789760735.306792:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 879 00000100:00100000:1.0:1789760735.306803:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508319360:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760735.306883:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd9ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508319360:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 82us (436us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760735.306891:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 879 00000100:00100000:1.0:1789760735.306908:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 880 00000100:00100000:1.0:1789760735.306915:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508319616:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760735.306939:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508319616:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 25us (439us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760735.306945:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 880 00000100:00100000:1.0:1789760735.306957:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 881 00000100:00100000:1.0:1789760735.306963:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd9b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508319488:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760735.306986:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd9b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508319488:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 23us (421us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760735.306991:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 881 00000100:00100000:1.0:1789760735.307006:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 882 00000100:00100000:1.0:1789760735.307012:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508319744:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760735.307035:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508319744:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 23us (442us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760735.307042:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 882 00000100:00100000:1.0:1789760735.307065:0:6460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508319232 00000100:00100000:1.0:1789760735.307069:0:6460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508319232 00000100:00100000:1.0:1789760735.307100:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 469 00000100:00080000:0.0:1789760735.307493:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789760735.307498:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760735.308088:0:6460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04989033850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2408:x1876699508319232:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760735.308146:0:6460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04989033850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2408:x1876699508319232:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 1039us (1609us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760735.308154:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 469 00000001:00080000:2.0:1789760735.358056:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972313 is committed 00002000:00010000:0.0:1789760735.358223:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20451328 -> 20455424 00010000:00010000:0.0:1789760735.358239:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54f96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.358252:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54f96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.358261:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54f96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.358268:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760735.358273:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54f96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.358323:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54f96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.358343:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54f96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.358350:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe54f96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20451328->20455423] (req 20451328->20455423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.358451:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508319104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57168us (57386us total) trans 4294972313 rc 0/0 00000100:00100000:0.0:1789760735.358460:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5025 00000100:00100000:0.0:1789760735.359219:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714336950736 time 714405976748 state 0x2 ref 0 00000100:00100000:3.0:1789760735.361827:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760735.361917:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508319872 00000100:00100000:1.0:1789760735.361927:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508319872 00000100:00100000:1.0:1789760735.361982:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714336950736 time 714405976748 state 0x0 ref 1 00000100:00100000:1.0:1789760735.361997:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714406237164 time 714405976748 state 0x1 ref 1 00000100:00100000:1.0:1789760735.362063:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714406237164 time 714466773237 state 0x0 ref 1 00000100:00100000:1.0:1789760735.362076:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508319872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760735.362131:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20455424, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:1.0:1789760735.362140:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.362155:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760735.362159:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.362165:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.362179:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.397962:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972314 is committed 00002000:00010000:1.0:1789760735.398048:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20455424 -> 20459520 00010000:00010000:1.0:1789760735.398063:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.398076:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.398086:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.398093:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760735.398098:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.398107:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.398137:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.398145:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54f9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20455424->20459519] (req 20455424->20459519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.398274:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508319872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36194us (36449us total) trans 4294972314 rc 0/0 00000100:00100000:1.0:1789760735.398329:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5026 00000100:00100000:1.0:1789760735.398340:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714406237164 time 714466773237 state 0x2 ref 0 00000100:00100000:1.0:1789760735.402227:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760735.402326:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508320000 00000100:00100000:2.0:1789760735.402335:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508320000 00000100:00100000:2.0:1789760735.402388:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714406237164 time 714466773237 state 0x0 ref 1 00000100:00100000:2.0:1789760735.402404:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714467033653 time 714466773237 state 0x1 ref 1 00000100:00100000:2.0:1789760735.402428:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714467033653 time 714507138738 state 0x0 ref 1 00000100:00100000:2.0:1789760735.402438:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508320000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760735.402489:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20459520, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:2.0:1789760735.402498:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54fa4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.402513:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.402517:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.402524:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54fa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.402537:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54fa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.429812:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972315 is committed 00002000:00010000:3.0:1789760735.429904:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20459520 -> 20463616 00010000:00010000:3.0:1789760735.429920:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54fa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.429934:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54fa4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.429943:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54fa4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.430230:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760735.430235:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54fa4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.430246:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54fa4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.430274:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54fa4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.430283:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe54fa4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20459520->20463615] (req 20459520->20463615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.430409:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508320000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27966us (28185us total) trans 4294972315 rc 0/0 00000100:00100000:3.0:1789760735.430420:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5027 00000100:00100000:3.0:1789760735.430430:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714467033653 time 714507138738 state 0x2 ref 0 00000100:00100000:0.0:1789760735.433964:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760735.434061:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508320128 00000100:00100000:1.0:1789760735.434071:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508320128 00000100:00100000:1.0:1789760735.434131:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714467033653 time 714507138738 state 0x0 ref 1 00000100:00100000:1.0:1789760735.434147:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714507399154 time 714507138738 state 0x1 ref 1 00000100:00100000:1.0:1789760735.434174:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714507399154 time 714538884987 state 0x0 ref 1 00000100:00100000:1.0:1789760735.434185:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508320128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760735.434231:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20463616, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:1.0:1789760735.434240:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54fab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.434255:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760735.434259:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.434265:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54fab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.434279:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54fab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.461381:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972316 is committed 00002000:00010000:0.0:1789760735.461687:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20463616 -> 20467712 00010000:00010000:0.0:1789760735.461702:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54fab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.461715:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54fab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.461724:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54fab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.461738:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760735.461743:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54fab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.461752:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54fab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.461772:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54fab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.461780:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe54fab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20463616->20467711] (req 20463616->20467711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.461875:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508320128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27686us (27914us total) trans 4294972316 rc 0/0 00000100:00100000:0.0:1789760735.461885:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5028 00000100:00100000:0.0:1789760735.461896:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714507399154 time 714538884987 state 0x2 ref 0 00000100:00100000:0.0:1789760735.466071:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760735.466164:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508320256 00000100:00100000:3.0:1789760735.466174:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508320256 00000100:00100000:3.0:1789760735.466227:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714507399154 time 714538884987 state 0x0 ref 1 00000100:00100000:3.0:1789760735.466242:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714539145403 time 714538884987 state 0x1 ref 1 00000100:00100000:3.0:1789760735.466263:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714539145403 time 714570973517 state 0x0 ref 1 00000100:00100000:3.0:1789760735.466275:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508320256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760735.466351:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20467712, mtime: 0x6aad94df, atime: 0x0, ctime: 0x6aad94df, blocks: 0xa007 00010000:00010000:3.0:1789760735.466361:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fb2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.466375:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.466380:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.466386:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.466399:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.492820:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972317 is committed 00002000:00010000:3.0:1789760735.492918:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20467712 -> 20471808 00002000:00010000:3.0:1789760735.492923:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760735 -> 1789760736 00002000:00010000:3.0:1789760735.492925:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760735 -> 1789760736 00010000:00010000:3.0:1789760735.492938:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.492958:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fb2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.492967:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.492974:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760735.492979:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.492988:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fb2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.493008:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fb2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.493015:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fb2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20467712->20471807] (req 20467712->20471807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.493880:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508320256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27600us (27812us total) trans 4294972317 rc 0/0 00000100:00100000:3.0:1789760735.493894:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5029 00000100:00100000:3.0:1789760735.493904:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714539145403 time 714570973517 state 0x2 ref 0 00000100:00100000:3.0:1789760735.497581:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760735.497719:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508320384 00000100:00100000:2.0:1789760735.497728:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508320384 00000100:00100000:2.0:1789760735.497781:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714539145403 time 714570973517 state 0x0 ref 1 00000100:00100000:2.0:1789760735.497795:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714571233933 time 714570973517 state 0x1 ref 1 00000100:00100000:2.0:1789760735.497823:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714571233933 time 714602533692 state 0x0 ref 1 00000100:00100000:2.0:1789760735.497834:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508320384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760735.497877:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20471808, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:2.0:1789760735.497886:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54fb9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.497900:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.497905:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.497910:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54fb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.497923:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54fb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.534744:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972318 is committed 00002000:00010000:0.0:1789760735.534922:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20471808 -> 20475904 00010000:00010000:0.0:1789760735.534942:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54fb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.534955:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54fb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.534971:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54fb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.534978:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760735.534983:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54fb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.534994:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54fb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.535017:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54fb9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.535026:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe54fb9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20471808->20475903] (req 20471808->20475903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760735.535442:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508320384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37604us (37865us total) trans 4294972318 rc 0/0 00000100:00100000:0.0:1789760735.535452:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5030 00000100:00100000:0.0:1789760735.535462:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714571233933 time 714602533692 state 0x2 ref 0 00000100:00100000:0.0:1789760735.545917:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760735.546008:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508320512 00000100:00100000:1.0:1789760735.546018:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508320512 00000100:00100000:1.0:1789760735.546075:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714571233933 time 714602533692 state 0x0 ref 1 00000100:00100000:1.0:1789760735.546090:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714602794108 time 714602533692 state 0x1 ref 1 00000100:00100000:1.0:1789760735.546130:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714602794108 time 714650840710 state 0x0 ref 1 00000100:00100000:1.0:1789760735.546142:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508320512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760735.546182:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20475904, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:1.0:1789760735.546191:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54fc0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.546207:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760735.546211:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.546217:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54fc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.546230:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54fc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.590765:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972319 is committed 00002000:00010000:3.0:1789760735.590992:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20475904 -> 20480000 00010000:00010000:3.0:1789760735.591008:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54fc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.591021:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54fc0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.591041:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54fc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.591048:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760735.591053:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54fc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.591062:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54fc0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.591082:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54fc0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.591090:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe54fc0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20475904->20479999] (req 20475904->20479999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.591193:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508320512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45047us (45279us total) trans 4294972319 rc 0/0 00000100:00100000:3.0:1789760735.591203:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5031 00000100:00100000:3.0:1789760735.591213:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714602794108 time 714650840710 state 0x2 ref 0 00000100:00100000:3.0:1789760735.599673:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760735.599826:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508320640 00000100:00100000:1.0:1789760735.599836:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508320640 00000100:00100000:1.0:1789760735.599887:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714602794108 time 714650840710 state 0x0 ref 1 00000100:00100000:1.0:1789760735.599902:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714651101126 time 714650840710 state 0x1 ref 1 00000100:00100000:1.0:1789760735.599931:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714651101126 time 714704640938 state 0x0 ref 1 00000100:00100000:1.0:1789760735.599943:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508320640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760735.599990:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20480000, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:1.0:1789760735.599999:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54fc7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.600016:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:1.0:1789760735.600020:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760735.600027:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54fc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.600041:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54fc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.641420:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972320 is committed 00002000:00010000:3.0:1789760735.641641:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20480000 -> 20484096 00010000:00010000:3.0:1789760735.641660:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54fc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.641673:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54fc7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.641681:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54fc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.641688:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760735.641693:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54fc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.641701:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54fc7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.641720:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54fc7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.641727:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe54fc7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20480000->20484095] (req 20480000->20484095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.641823:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508320640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41877us (42154us total) trans 4294972320 rc 0/0 00000100:00100000:3.0:1789760735.641836:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5032 00000100:00100000:3.0:1789760735.641846:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714651101126 time 714704640938 state 0x2 ref 0 00000100:00100000:1.0:1789760735.648991:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760735.649084:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508320768 00000100:00100000:0.0:1789760735.649093:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508320768 00000100:00100000:0.0:1789760735.649157:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714651101126 time 714704640938 state 0x0 ref 1 00000100:00100000:0.0:1789760735.649172:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714704901354 time 714704640938 state 0x1 ref 1 00000100:00100000:0.0:1789760735.649204:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714704901354 time 714753914873 state 0x0 ref 1 00000100:00100000:0.0:1789760735.649218:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508320768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760735.649271:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20484096, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:0.0:1789760735.649280:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54fce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.649426:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760735.649431:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760735.649437:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54fce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760735.649452:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54fce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760735.682205:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20484096 -> 20488192 00010000:00010000:1.0:1789760735.682226:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54fce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.682240:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54fce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.682255:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54fce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.682263:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760735.682268:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54fce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.682277:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54fce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.682316:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54fce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.682324:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe54fce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20484096->20488191] (req 20484096->20488191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.682420:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508320768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33199us (33434us total) trans 4294972321 rc 0/0 00000100:00100000:1.0:1789760735.682431:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5033 00000100:00100000:1.0:1789760735.682441:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714704901354 time 714753914873 state 0x2 ref 0 00000001:00080000:3.0:1789760735.682572:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972321 is committed 00000100:00100000:0.0:1789760735.686677:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760735.686787:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508320896 00000100:00100000:3.0:1789760735.686798:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508320896 00000100:00100000:3.0:1789760735.686855:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714704901354 time 714753914873 state 0x0 ref 1 00000100:00100000:3.0:1789760735.686870:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714754175289 time 714753914873 state 0x1 ref 1 00000100:00100000:3.0:1789760735.686953:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714754175289 time 714791653688 state 0x0 ref 1 00000100:00100000:3.0:1789760735.686966:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508320896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760735.687016:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20488192, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760735.687025:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fd5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.687050:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.687055:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.687062:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.687076:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.717629:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972322 is committed 00002000:00010000:3.0:1789760735.717773:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20488192 -> 20492288 00010000:00010000:3.0:1789760735.717788:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.717800:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.717809:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.717817:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760735.717822:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.717830:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fd5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.717849:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fd5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.717857:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fd5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20488192->20492287] (req 20488192->20492287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.717947:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508320896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30979us (31275us total) trans 4294972322 rc 0/0 00000100:00100000:3.0:1789760735.717957:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5034 00000100:00100000:3.0:1789760735.717967:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714754175289 time 714791653688 state 0x2 ref 0 00000100:00100000:2.0:1789760735.727192:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760735.727840:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508321024 00000100:00100000:3.0:1789760735.727850:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508321024 00000100:00100000:3.0:1789760735.727906:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714754175289 time 714791653688 state 0x0 ref 1 00000100:00100000:3.0:1789760735.727921:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714791914104 time 714791653688 state 0x1 ref 1 00000100:00100000:3.0:1789760735.727995:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714791914104 time 714832703978 state 0x0 ref 1 00000100:00100000:3.0:1789760735.728010:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508321024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760735.728056:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20492288, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760735.728065:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fdc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.728080:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.728084:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.728091:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.728104:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760735.751266:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20492288 -> 20496384 00010000:00010000:3.0:1789760735.751302:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.751314:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fdc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.751323:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.751330:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760735.751335:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.751344:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fdc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.751364:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fdc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.751371:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fdc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20492288->20496383] (req 20492288->20496383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.751473:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508321024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23453us (24279us total) trans 4294972323 rc 0/0 00000100:00100000:3.0:1789760735.751484:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5035 00000100:00100000:3.0:1789760735.751494:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714791914104 time 714832703978 state 0x2 ref 0 00000001:00080000:2.0:1789760735.752486:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972323 is committed 00000100:00100000:1.0:1789760735.755931:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760735.756032:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508321152 00000100:00100000:2.0:1789760735.756042:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508321152 00000100:00100000:2.0:1789760735.756095:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714791914104 time 714832703978 state 0x0 ref 1 00000100:00100000:2.0:1789760735.756110:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714832964394 time 714832703978 state 0x1 ref 1 00000100:00100000:2.0:1789760735.756151:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714832964394 time 714860860646 state 0x0 ref 1 00000100:00100000:2.0:1789760735.756163:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508321152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760735.756207:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20496384, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:2.0:1789760735.756216:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54fe3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.756230:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760735.756235:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760735.756241:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54fe3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.756260:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54fe3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.793251:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972324 is committed 00002000:00010000:3.0:1789760735.793444:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20496384 -> 20500480 00010000:00010000:3.0:1789760735.793458:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54fe3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.793487:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54fe3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.793496:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54fe3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.793503:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760735.793508:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54fe3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.793518:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54fe3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.793537:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54fe3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.793544:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe54fe3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20496384->20500479] (req 20496384->20500479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.793639:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508321152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37473us (37713us total) trans 4294972324 rc 0/0 00000100:00100000:3.0:1789760735.793649:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5036 00000100:00100000:3.0:1789760735.793659:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714832964394 time 714860860646 state 0x2 ref 0 00000100:00100000:1.0:1789760735.803257:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760735.803404:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508321280 00000100:00100000:3.0:1789760735.803415:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508321280 00000100:00100000:3.0:1789760735.803469:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714832964394 time 714860860646 state 0x0 ref 1 00000100:00100000:3.0:1789760735.803484:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714861121062 time 714860860646 state 0x1 ref 1 00000100:00100000:3.0:1789760735.803513:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714861121062 time 714908223890 state 0x0 ref 1 00000100:00100000:3.0:1789760735.803525:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508321280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760735.803574:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20500480, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760735.803584:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.803600:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.803604:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.803611:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.803625:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.822359:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972325 is committed 00002000:00010000:3.0:1789760735.822527:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20500480 -> 20504576 00010000:00010000:3.0:1789760735.822542:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.822552:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.822566:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.822573:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760735.822578:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.822587:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.822605:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.822614:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe54fea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20500480->20504575] (req 20500480->20504575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.822799:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508321280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19270us (19546us total) trans 4294972325 rc 0/0 00000100:00100000:3.0:1789760735.822811:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5037 00000100:00100000:3.0:1789760735.822821:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714861121062 time 714908223890 state 0x2 ref 0 00000100:00100000:0.0:1789760735.826498:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760735.826579:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508321408 00000100:00100000:3.0:1789760735.826589:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508321408 00000100:00100000:3.0:1789760735.826643:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714861121062 time 714908223890 state 0x0 ref 1 00000100:00100000:3.0:1789760735.826659:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714908484306 time 714908223890 state 0x1 ref 1 00000100:00100000:3.0:1789760735.826694:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714908484306 time 714931403955 state 0x0 ref 1 00000100:00100000:3.0:1789760735.826707:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508321408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760735.826752:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20504576, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760735.826761:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.826777:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.826782:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.826788:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.826803:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760735.850041:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972326 is committed 00002000:00010000:3.0:1789760735.850167:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20504576 -> 20508672 00010000:00010000:3.0:1789760735.850192:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.850206:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.850216:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.850223:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760735.850230:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.850239:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.850260:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.850268:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20504576->20508671] (req 20504576->20508671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.853278:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508321408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23751us (23969us total) trans 4294972326 rc 0/0 00000100:00100000:3.0:1789760735.853318:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5038 00000100:00100000:3.0:1789760735.853330:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714908484306 time 714931403955 state 0x2 ref 0 00000100:00100000:1.0:1789760735.859342:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760735.862424:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508321536 00000100:00100000:3.0:1789760735.862439:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508321536 00000100:00100000:3.0:1789760735.862513:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714908484306 time 714931403955 state 0x0 ref 1 00000100:00100000:3.0:1789760735.862529:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714931664371 time 714931403955 state 0x1 ref 1 00000100:00100000:3.0:1789760735.862568:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714931664371 time 714967277810 state 0x0 ref 1 00000100:00100000:3.0:1789760735.862581:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508321536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760735.862630:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20508672, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760735.862640:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.862656:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760735.862660:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.862666:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.862679:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760735.891723:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972327 is committed 00002000:00010000:3.0:1789760735.891787:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20508672 -> 20512768 00010000:00010000:3.0:1789760735.891800:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.891812:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.891821:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.891827:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760735.891831:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.891839:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.891857:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.891864:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54ff8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20508672->20512767] (req 20508672->20512767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.893502:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508321536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30913us (34159us total) trans 4294972327 rc 0/0 00000100:00100000:3.0:1789760735.893515:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5039 00000100:00100000:3.0:1789760735.893530:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714931664371 time 714967277810 state 0x2 ref 0 00000100:00100000:1.0:1789760735.897615:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760735.898381:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508321664 00000100:00100000:3.0:1789760735.898392:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508321664 00000100:00100000:3.0:1789760735.898449:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714931664371 time 714967277810 state 0x0 ref 1 00000100:00100000:3.0:1789760735.898466:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714967538226 time 714967277810 state 0x1 ref 1 00000100:00100000:3.0:1789760735.898541:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714967538226 time 715003250252 state 0x0 ref 1 00000100:00100000:3.0:1789760735.898553:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508321664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760735.898600:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20512768, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760735.898609:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54fff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.898624:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760735.898628:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.898635:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54fff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.898651:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54fff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.927215:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972328 is committed 00002000:00010000:2.0:1789760735.927526:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20512768 -> 20516864 00010000:00010000:2.0:1789760735.927542:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54fff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.927554:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54fff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.927564:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54fff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.927571:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760735.927576:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54fff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.927585:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54fff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.927604:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54fff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760735.927612:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe54fff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20512768->20516863] (req 20512768->20516863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760735.927705:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508321664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29149us (30095us total) trans 4294972328 rc 0/0 00000100:00100000:2.0:1789760735.927716:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5040 00000100:00100000:2.0:1789760735.927725:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714967538226 time 715003250252 state 0x2 ref 0 00000100:00100000:1.0:1789760735.931245:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760735.931325:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508321792 00000100:00100000:3.0:1789760735.931335:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508321792 00000100:00100000:3.0:1789760735.931385:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 714967538226 time 715003250252 state 0x0 ref 1 00000100:00100000:3.0:1789760735.931399:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715003510668 time 715003250252 state 0x1 ref 1 00000100:00100000:3.0:1789760735.931423:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715003510668 time 715036133734 state 0x0 ref 1 00000100:00100000:3.0:1789760735.931433:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508321792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760735.931485:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20516864, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760735.931493:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55006 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.931508:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760735.931512:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.931518:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55006 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.931531:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55006 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760735.960760:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972329 is committed 00002000:00010000:3.0:1789760735.960854:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20516864 -> 20520960 00010000:00010000:3.0:1789760735.960867:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55006 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.960880:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55006 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.960888:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55006 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.960894:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760735.960899:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55006 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.960907:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55006 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.960925:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55006 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.960932:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55006 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20516864->20520959] (req 20516864->20520959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760735.961049:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508321792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29612us (29807us total) trans 4294972329 rc 0/0 00000100:00100000:3.0:1789760735.961062:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5041 00000100:00100000:3.0:1789760735.961072:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715003510668 time 715036133734 state 0x2 ref 0 00000100:00100000:1.0:1789760735.964761:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760735.964875:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508321920 00000100:00100000:3.0:1789760735.964886:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508321920 00000100:00100000:3.0:1789760735.964946:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715003510668 time 715036133734 state 0x0 ref 1 00000100:00100000:3.0:1789760735.964962:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715036394150 time 715036133734 state 0x1 ref 1 00000100:00100000:3.0:1789760735.964992:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715036394150 time 715069702840 state 0x0 ref 1 00000100:00100000:3.0:1789760735.965005:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508321920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760735.965053:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20520960, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760735.965063:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5500d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.965080:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760735.965084:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760735.965090:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5500d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760735.965104:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5500d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760735.993325:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20520960 -> 20525056 00010000:00010000:1.0:1789760735.993344:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5500d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.993359:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5500d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.993368:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5500d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.993376:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760735.993381:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5500d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.993390:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5500d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.993410:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5500d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760735.993417:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5500d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20520960->20525055] (req 20520960->20525055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760735.993584:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508321920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28577us (28828us total) trans 4294972330 rc 0/0 00000100:00100000:1.0:1789760735.993596:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5042 00000100:00100000:1.0:1789760735.993605:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715036394150 time 715069702840 state 0x2 ref 0 00000001:00080000:1.0:1789760735.993642:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972330 is committed 00010000:00010000:1.0:1789760735.998562:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760736.001324:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760736.001411:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508322048 00000100:00100000:2.0:1789760736.001422:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508322048 00000100:00100000:2.0:1789760736.001484:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715036394150 time 715069702840 state 0x0 ref 1 00000100:00100000:2.0:1789760736.001499:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715069963256 time 715069702840 state 0x1 ref 1 00000100:00100000:2.0:1789760736.001532:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715069963256 time 715106241980 state 0x0 ref 1 00000100:00100000:2.0:1789760736.001544:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508322048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760736.001591:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20525056, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:2.0:1789760736.001600:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55014 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.001616:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760736.001620:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.001626:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55014 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.001640:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55014 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.033766:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972331 is committed 00002000:00010000:2.0:1789760736.033849:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20525056 -> 20529152 00010000:00010000:2.0:1789760736.033865:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55014 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.033879:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55014 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.033890:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55014 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.033898:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760736.033904:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55014 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.033914:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55014 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.033935:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55014 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.033944:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55014 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20525056->20529151] (req 20525056->20529151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.034184:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508322048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32635us (32894us total) trans 4294972331 rc 0/0 00000100:00100000:2.0:1789760736.034195:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5043 00000100:00100000:2.0:1789760736.034205:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715069963256 time 715106241980 state 0x2 ref 0 00000100:00100000:2.0:1789760736.039072:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760736.039153:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508322176 00000100:00100000:3.0:1789760736.039164:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508322176 00000100:00100000:3.0:1789760736.039228:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715069963256 time 715106241980 state 0x0 ref 1 00000100:00100000:3.0:1789760736.039244:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715106502396 time 715106241980 state 0x1 ref 1 00000100:00100000:3.0:1789760736.039357:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715106502396 time 715144066494 state 0x0 ref 1 00000100:00100000:3.0:1789760736.039369:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d46050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508322176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760736.039414:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20529152, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760736.039422:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5501b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.039437:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.039441:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.039447:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5501b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.039461:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5501b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.070240:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972332 is committed 00002000:00010000:1.0:1789760736.070348:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20529152 -> 20533248 00010000:00010000:1.0:1789760736.070364:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5501b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.070376:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5501b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.070385:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5501b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.070392:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760736.070397:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5501b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.070405:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5501b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.070433:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5501b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.070441:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5501b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20529152->20533247] (req 20529152->20533247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.070550:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d46050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508322176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31177us (31484us total) trans 4294972332 rc 0/0 00000100:00100000:1.0:1789760736.070560:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5044 00000100:00100000:1.0:1789760736.070569:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715106502396 time 715144066494 state 0x2 ref 0 00000100:00100000:1.0:1789760736.085446:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760736.085536:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508322304 00000100:00100000:3.0:1789760736.085546:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508322304 00000100:00100000:3.0:1789760736.085600:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715106502396 time 715144066494 state 0x0 ref 1 00000100:00100000:3.0:1789760736.085614:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715144326910 time 715144066494 state 0x1 ref 1 00000100:00100000:3.0:1789760736.085643:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715144326910 time 715190354240 state 0x0 ref 1 00000100:00100000:3.0:1789760736.085655:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508322304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760736.085701:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20533248, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760736.085709:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55022 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.085724:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.085728:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.085734:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55022 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.085747:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55022 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.121668:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972333 is committed 00002000:00010000:1.0:1789760736.121796:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20533248 -> 20537344 00010000:00010000:1.0:1789760736.121812:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55022 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.121826:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55022 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.121835:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55022 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.121842:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760736.121847:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55022 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.121856:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55022 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.121881:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55022 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.121889:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55022 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20533248->20537343] (req 20533248->20537343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.121984:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508322304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36325us (36542us total) trans 4294972333 rc 0/0 00000100:00100000:1.0:1789760736.121995:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5045 00000100:00100000:1.0:1789760736.122005:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715144326910 time 715190354240 state 0x2 ref 0 00000100:00100000:0.0:1789760736.127124:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760736.127149:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049891fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534066560:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760736.127223:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760736.128135:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534066560 00000100:00100000:1.0:1789760736.128146:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534066560 00000100:00100000:1.0:1789760736.128204:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:1.0:1789760736.128214:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534066560:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760736.128249:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760736.128324:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534066560:12345-0@lo:13:osp-pre-1-1.0 Request processed in 111us (1100us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760736.128331:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:1.0:1789760736.128379:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049891fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534066560:0@lo:13:osp-pre-1-1.0 00000100:00080000:0.0:1789760736.128439:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760736.129650:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd50700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3472:1876699534066688:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.129705:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760736.129746:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd53100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534067200:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.129772:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760736.129791:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c32ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534067712:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.129813:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760736.129835:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1789760736.129841:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd50a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3472:1876699534066816:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.129965:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760736.129986:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd51f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3472:1876699534067328:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.130006:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760736.130332:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534067200 00000100:00100000:0.0:1789760736.130340:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534067200 00000100:00100000:0.0:1789760736.130381:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 426 00000100:00100000:0.0:1789760736.130389:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534067200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760736.130437:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880fa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534067200:12345-0@lo:400:kworker.0 Request processed in 48us (663us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760736.130443:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 426 00000100:00100000:0.0:1789760736.130464:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534066688 00000100:00100000:0.0:1789760736.130467:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534066688 00000100:00100000:0.0:1789760736.130491:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:0.0:1789760736.130497:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04989030850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3472:x1876699534066688:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760736.130526:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04989030850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3472:x1876699534066688:12345-0@lo:400:kworker.0 Request processed in 29us (821us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760736.130532:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00100000:0.0:1789760736.130560:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd51500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3475:1876699534067072:0@lo:400:kworker.0 00000100:00100000:0.0:1789760736.130591:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b645500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534068352:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760736.130615:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760736.130636:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760736.130642:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04985468380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534068224:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760736.130664:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760736.130695:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534066944:0@lo:400:kworker.0 00000100:00100000:0.0:1789760736.130718:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760736.130734:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534067456:0@lo:400:kworker.0 00000100:00100000:0.0:1789760736.130760:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534066944 00000100:00100000:0.0:1789760736.130764:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534066944 00000100:00100000:0.0:1789760736.130787:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 427 00000100:00100000:0.0:1789760736.130794:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534066944:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760736.130830:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534066944:12345-0@lo:400:kworker.0 Request processed in 36us (111us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760736.130836:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 427 00000100:00100000:3.0:1789760736.131093:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c32f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3472:1876699534067840:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.131129:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760736.131464:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534068352 00000100:00100000:1.0:1789760736.131469:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534068352 00000100:00100000:1.0:1789760736.131502:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534068224 00000100:00100000:1.0:1789760736.131505:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534068224 00000100:00100000:1.0:1789760736.131524:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:1.0:1789760736.131531:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534068352:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760736.131547:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526188544 pending 0 free 3837788160 avail 3806330880 00000100:00100000:1.0:1789760736.131589:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534068352:12345-0@lo:13:osp-pre-0-1.0 Request processed in 58us (973us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760736.131596:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00100000:1.0:1789760736.131610:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:1.0:1789760736.131616:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534068224:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760736.131626:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760736.131649:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534068224:12345-0@lo:13:osp-pre-1-0.0 Request processed in 34us (985us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760736.131654:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000100:00100000:1.0:1789760736.131693:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04985468380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534068224:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760736.131705:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b645500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534068352:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1789760736.135425:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760736.135435:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760736.135995:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd50700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3472:1876699534066688:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.136024:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd53100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534067200:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.136061:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534067712 00000100:00100000:3.0:1789760736.136067:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534067712 00000100:00100000:3.0:1789760736.136137:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 223 00000100:00100000:3.0:1789760736.136146:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534067712:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.136216:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534067712:12345-0@lo:400:kworker.0 Request processed in 71us (6402us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760736.136224:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 223 00000100:00100000:2.0:1789760736.137878:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534066816 00000100:00100000:2.0:1789760736.137889:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534066816 00000100:00100000:2.0:1789760736.138119:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534067328 00000100:00100000:2.0:1789760736.138124:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534067328 00000100:00100000:3.0:1789760736.138175:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534067840 00000100:00100000:3.0:1789760736.138179:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534067840 00000100:00100000:1.0:1789760736.138222:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 883 00000100:00100000:1.0:1789760736.138234:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3472:x1876699534066816:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760736.138334:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880fad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3472:x1876699534066816:12345-0@lo:400:kworker.0 Request processed in 102us (8368us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760736.138342:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 883 00000100:00100000:1.0:1789760736.138357:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 884 00000100:00100000:1.0:1789760736.138363:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3472:x1876699534067328:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760736.138391:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880f8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3472:x1876699534067328:12345-0@lo:400:kworker.0 Request processed in 28us (8384us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760736.138396:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 884 00000100:00100000:1.0:1789760736.138405:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 885 00000100:00100000:1.0:1789760736.138411:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880fb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3472:x1876699534067840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760736.138438:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880fb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3472:x1876699534067840:12345-0@lo:400:kworker.0 Request processed in 27us (7309us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760736.138443:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 885 00000100:00100000:3.0:1789760736.142265:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.142453:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760736.142494:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508322432 00000100:00100000:2.0:1789760736.142504:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508322432 00000100:00100000:2.0:1789760736.142561:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715144326910 time 715190354240 state 0x0 ref 1 00000100:00100000:2.0:1789760736.142577:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715190614656 time 715190354240 state 0x1 ref 1 00000100:00100000:2.0:1789760736.142608:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715190614656 time 715247318664 state 0x0 ref 1 00000100:00100000:2.0:1789760736.142620:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508322432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760736.142666:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20537344, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:2.0:1789760736.142675:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55029 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.142691:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760736.142695:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.142701:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55029 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.142714:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55029 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.145444:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534067584:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.145504:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.145515:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760736.145531:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048880f8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534068096:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.145555:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760736.146282:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534067072 00000100:00100000:0.0:1789760736.146301:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534067072 00000100:00100000:0.0:1789760736.146354:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 886 00000100:00100000:0.0:1789760736.146363:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3475:x1876699534067072:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760736.146412:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880f9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3475:x1876699534067072:12345-0@lo:400:kworker.0 Request processed in 50us (4155us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760736.146418:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 886 00000100:00100000:3.0:1789760736.146893:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd51500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3475:1876699534067072:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.146910:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760736.146917:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049891fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534068480:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760736.146948:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760736.146989:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534067584 00000100:00100000:3.0:1789760736.146993:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534067584 00000100:00100000:2.0:1789760736.147173:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048880fbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534067968:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.147202:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760736.147230:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534066944:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.147251:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534067456 00000100:00100000:2.0:1789760736.147254:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534067456 00000100:00100000:2.0:1789760736.147283:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534067968 00000100:00100000:2.0:1789760736.147301:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534067968 00000100:00100000:2.0:1789760736.147321:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 887 00000100:00100000:2.0:1789760736.147328:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534067456:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.147372:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534067456:12345-0@lo:400:kworker.0 Request processed in 44us (1859us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760736.147378:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 887 00000100:00100000:2.0:1789760736.147391:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 888 00000100:00100000:2.0:1789760736.147397:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534067968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.147424:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534067968:12345-0@lo:400:kworker.0 Request processed in 27us (221us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760736.147430:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 888 00000100:00100000:2.0:1789760736.147457:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534067456:0@lo:400:kworker.0 00000100:00100000:2.0:1789760736.147467:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048880fbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534067968:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.148138:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534068096 00000100:00100000:3.0:1789760736.148143:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534068096 00000100:00100000:3.0:1789760736.148174:0:8102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00000100:00100000:3.0:1789760736.148181:0:8102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534067584:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.148228:0:8102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534067584:12345-0@lo:400:kworker.0 Request processed in 48us (2723us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760736.148235:0:8102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00100000:3.0:1789760736.148249:0:8102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:3.0:1789760736.148254:0:8102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534068096:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.148278:0:8102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534068096:12345-0@lo:400:kworker.0 Request processed in 25us (2723us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760736.148298:0:8102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00080000:3.0:1789760736.148320:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760736.148324:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760736.148327:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760736.148351:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd50a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3472:1876699534066816:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.148362:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd51f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3472:1876699534067328:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.148373:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c32f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3472:1876699534067840:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.148383:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c32ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534067712:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.148411:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534067584:0@lo:400:kworker.0 00000100:00100000:3.0:1789760736.148421:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048880f8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534068096:0@lo:400:kworker.0 00000100:00100000:0.0:1789760736.148800:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534068480 00000100:00100000:0.0:1789760736.148804:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534068480 00000100:00100000:0.0:1789760736.148840:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 367 00000100:00100000:0.0:1789760736.148848:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534068480:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760736.148863:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3837788160 avail 3806330880 00000100:00100000:0.0:1789760736.148900:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534068480:12345-0@lo:13:osp-pre-0-0.0 Request processed in 53us (1951us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760736.148906:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 367 00000100:00100000:3.0:1789760736.150822:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049891fc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534068480:0@lo:13:osp-pre-0-0.0 00000001:00080000:1.0:1789760736.182331:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972334 is committed 00002000:00010000:0.0:1789760736.182493:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20537344 -> 20541440 00010000:00010000:0.0:1789760736.182510:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55029 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.182528:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55029 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.182538:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55029 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.182545:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760736.182550:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55029 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.182559:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55029 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.182579:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55029 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.182587:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55029 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20537344->20541439] (req 20537344->20541439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.182697:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508322432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40074us (40242us total) trans 4294972334 rc 0/0 00000100:00100000:0.0:1789760736.182707:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5046 00000100:00100000:0.0:1789760736.182716:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715190614656 time 715247318664 state 0x2 ref 0 00000100:00100000:2.0:1789760736.185129:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760736.185402:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508322560 00000100:00100000:1.0:1789760736.185411:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508322560 00000100:00100000:1.0:1789760736.185476:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715190614656 time 715247318664 state 0x0 ref 1 00000100:00100000:1.0:1789760736.185496:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715247579080 time 715247318664 state 0x1 ref 1 00000100:00100000:1.0:1789760736.185522:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715247579080 time 715290232862 state 0x0 ref 1 00000100:00100000:1.0:1789760736.185532:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508322560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760736.185579:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20541440, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:1.0:1789760736.185588:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55030 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.185603:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760736.185607:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.185613:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55030 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.185626:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55030 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.205558:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972335 is committed 00002000:00010000:0.0:1789760736.205609:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20541440 -> 20545536 00010000:00010000:0.0:1789760736.205624:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55030 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.205636:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55030 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.205645:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55030 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.205652:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760736.205657:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55030 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.205666:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55030 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.205684:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55030 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.205692:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55030 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20541440->20545535] (req 20541440->20545535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.205786:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508322560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20249us (20669us total) trans 4294972335 rc 0/0 00000100:00100000:0.0:1789760736.205795:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5047 00000100:00100000:0.0:1789760736.206137:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715247579080 time 715290232862 state 0x2 ref 0 00000100:00100000:1.0:1789760736.214853:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760736.214990:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508322688 00000100:00100000:3.0:1789760736.214999:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508322688 00000100:00100000:3.0:1789760736.215051:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715247579080 time 715290232862 state 0x0 ref 1 00000100:00100000:3.0:1789760736.215071:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715290493278 time 715290232862 state 0x1 ref 1 00000100:00100000:3.0:1789760736.215102:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715290493278 time 715319812411 state 0x0 ref 1 00000100:00100000:3.0:1789760736.215123:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508322688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760736.215173:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20545536, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760736.215182:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55037 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.215196:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.215200:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.215206:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55037 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.215218:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55037 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.270202:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972336 is committed 00002000:00010000:3.0:1789760736.270282:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20545536 -> 20549632 00010000:00010000:3.0:1789760736.270359:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55037 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.270371:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55037 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.270379:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55037 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.270391:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760736.270396:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55037 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.270406:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55037 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.270424:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55037 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.270431:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55037 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20545536->20549631] (req 20545536->20549631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.270618:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508322688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55501us (55769us total) trans 4294972336 rc 0/0 00000100:00100000:3.0:1789760736.270628:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5048 00000100:00100000:3.0:1789760736.270638:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715290493278 time 715319812411 state 0x2 ref 0 00000100:00100000:2.0:1789760736.273932:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760736.274120:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508322816 00000100:00100000:0.0:1789760736.274130:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508322816 00000100:00100000:0.0:1789760736.274183:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715290493278 time 715319812411 state 0x0 ref 1 00000100:00100000:0.0:1789760736.274197:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715320072827 time 715319812411 state 0x1 ref 1 00000100:00100000:0.0:1789760736.274261:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715320072827 time 715378970996 state 0x0 ref 1 00000100:00100000:0.0:1789760736.274273:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508322816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760736.274355:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20549632, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:0.0:1789760736.274364:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5503e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.274378:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760736.274382:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.274389:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5503e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.274402:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5503e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760736.303625:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20549632 -> 20553728 00010000:00010000:3.0:1789760736.303645:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5503e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.303659:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5503e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.303674:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5503e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.303682:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760736.303687:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5503e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.303696:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5503e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.303715:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5503e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.303723:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5503e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20549632->20553727] (req 20549632->20553727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.303849:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508322816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29572us (29921us total) trans 4294972337 rc 0/0 00000100:00100000:3.0:1789760736.303860:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5049 00000100:00100000:3.0:1789760736.303869:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715320072827 time 715378970996 state 0x2 ref 0 00000001:00080000:0.0:1789760736.303953:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972337 is committed 00000100:00100000:2.0:1789760736.308249:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760736.308573:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508322944 00000100:00100000:0.0:1789760736.308583:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508322944 00000100:00100000:0.0:1789760736.308841:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715320072827 time 715378970996 state 0x0 ref 1 00000100:00100000:0.0:1789760736.308860:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715379231412 time 715378970996 state 0x1 ref 1 00000100:00100000:0.0:1789760736.308921:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715379231412 time 715413631201 state 0x0 ref 1 00000100:00100000:0.0:1789760736.308934:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508322944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760736.308981:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20553728, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:0.0:1789760736.308991:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55045 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.309007:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760736.309011:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.309017:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55045 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.309042:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55045 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.336250:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972338 is committed 00002000:00010000:1.0:1789760736.336440:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20553728 -> 20557824 00010000:00010000:1.0:1789760736.336455:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55045 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.336479:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55045 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.336489:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55045 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.336496:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760736.336501:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55045 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.336510:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55045 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.336530:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55045 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.336538:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55045 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20553728->20557823] (req 20553728->20557823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.336633:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508322944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27696us (28388us total) trans 4294972338 rc 0/0 00000100:00100000:1.0:1789760736.336642:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5050 00000100:00100000:1.0:1789760736.336652:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715379231412 time 715413631201 state 0x2 ref 0 00000100:00100000:1.0:1789760736.342708:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760736.342858:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508323072 00000100:00100000:3.0:1789760736.342869:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508323072 00000100:00100000:3.0:1789760736.342924:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715379231412 time 715413631201 state 0x0 ref 1 00000100:00100000:3.0:1789760736.342941:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715413891617 time 715413631201 state 0x1 ref 1 00000100:00100000:3.0:1789760736.342977:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715413891617 time 715447686804 state 0x0 ref 1 00000100:00100000:3.0:1789760736.342990:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508323072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760736.343051:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20557824, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760736.343060:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5504c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.343077:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.343082:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.343088:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5504c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.343102:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5504c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760736.374965:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972339 is committed 00002000:00010000:3.0:1789760736.376710:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20557824 -> 20561920 00010000:00010000:3.0:1789760736.376727:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5504c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.376740:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5504c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.376749:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5504c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.376757:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760736.376762:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5504c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.376771:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5504c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.376790:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5504c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.376797:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5504c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20557824->20561919] (req 20557824->20561919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.376949:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508323072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33955us (34245us total) trans 4294972339 rc 0/0 00000100:00100000:3.0:1789760736.376961:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5051 00000100:00100000:3.0:1789760736.376971:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715413891617 time 715447686804 state 0x2 ref 0 00000100:00100000:2.0:1789760736.383279:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760736.383392:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508323200 00000100:00100000:3.0:1789760736.383402:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508323200 00000100:00100000:3.0:1789760736.383457:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715413891617 time 715447686804 state 0x0 ref 1 00000100:00100000:3.0:1789760736.383482:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715447947220 time 715447686804 state 0x1 ref 1 00000100:00100000:3.0:1789760736.383510:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715447947220 time 715488221002 state 0x0 ref 1 00000100:00100000:3.0:1789760736.383522:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508323200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760736.383569:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20561920, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760736.383578:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55053 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.383593:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.383597:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.383603:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55053 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.383618:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55053 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760736.414786:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20561920 -> 20566016 00010000:00010000:3.0:1789760736.414805:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55053 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.414819:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55053 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.414828:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55053 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.414835:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760736.414840:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55053 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.414850:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55053 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.414876:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55053 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.414884:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55053 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20561920->20566015] (req 20561920->20566015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.414980:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508323200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31454us (31706us total) trans 4294972340 rc 0/0 00000100:00100000:3.0:1789760736.414991:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5052 00000100:00100000:3.0:1789760736.415001:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715447947220 time 715488221002 state 0x2 ref 0 00000001:00080000:1.0:1789760736.415614:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972340 is committed 00000100:00100000:2.0:1789760736.418394:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760736.418502:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508323328 00000100:00100000:1.0:1789760736.418512:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508323328 00000100:00100000:1.0:1789760736.418566:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715447947220 time 715488221002 state 0x0 ref 1 00000100:00100000:1.0:1789760736.418581:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715488481418 time 715488221002 state 0x1 ref 1 00000100:00100000:1.0:1789760736.418612:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715488481418 time 715523322532 state 0x0 ref 1 00000100:00100000:1.0:1789760736.418626:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508323328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760736.418671:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20566016, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:1.0:1789760736.418680:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5505a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.418694:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:1.0:1789760736.418699:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760736.418705:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5505a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.418720:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5505a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.441789:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972341 is committed 00002000:00010000:1.0:1789760736.441891:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20566016 -> 20570112 00010000:00010000:1.0:1789760736.441906:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5505a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.441918:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5505a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.441927:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5505a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.441935:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760736.441940:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5505a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.441949:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5505a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.441969:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5505a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760736.441977:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5505a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20566016->20570111] (req 20566016->20570111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760736.442196:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508323328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23566us (23804us total) trans 4294972341 rc 0/0 00000100:00100000:1.0:1789760736.442207:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5053 00000100:00100000:1.0:1789760736.442217:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715488481418 time 715523322532 state 0x2 ref 0 00000100:00100000:2.0:1789760736.446241:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760736.448148:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508323456 00000100:00100000:3.0:1789760736.448162:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508323456 00000100:00100000:3.0:1789760736.448223:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715488481418 time 715523322532 state 0x0 ref 1 00000100:00100000:3.0:1789760736.448237:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715523582948 time 715523322532 state 0x1 ref 1 00000100:00100000:3.0:1789760736.448267:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715523582948 time 715552977599 state 0x0 ref 1 00000100:00100000:3.0:1789760736.448281:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508323456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760736.448348:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20570112, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760736.448356:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55061 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.448372:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.448377:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.448384:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55061 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.448398:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55061 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.474744:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972342 is committed 00002000:00010000:3.0:1789760736.474747:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20570112 -> 20574208 00010000:00010000:3.0:1789760736.474764:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55061 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.474777:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55061 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.474787:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55061 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.474884:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760736.474889:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55061 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.474898:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55061 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.474917:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55061 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.474928:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55061 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20570112->20574207] (req 20570112->20574207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.475022:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508323456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26739us (28785us total) trans 4294972342 rc 0/0 00000100:00100000:3.0:1789760736.475038:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5054 00000100:00100000:3.0:1789760736.475048:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715523582948 time 715552977599 state 0x2 ref 0 00000100:00100000:2.0:1789760736.478099:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760736.478206:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508323584 00000100:00100000:3.0:1789760736.478216:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508323584 00000100:00100000:3.0:1789760736.478272:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715523582948 time 715552977599 state 0x0 ref 1 00000100:00100000:3.0:1789760736.478372:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715553238015 time 715552977599 state 0x1 ref 1 00000100:00100000:3.0:1789760736.478433:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715553238015 time 715583142974 state 0x0 ref 1 00000100:00100000:3.0:1789760736.478447:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508323584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760736.478504:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20574208, mtime: 0x6aad94e0, atime: 0x0, ctime: 0x6aad94e0, blocks: 0xa007 00010000:00010000:3.0:1789760736.478513:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55068 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.478528:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.478533:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.478540:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55068 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.478564:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55068 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760736.502996:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20574208 -> 20578304 00002000:00010000:3.0:1789760736.503006:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760736 -> 1789760737 00002000:00010000:3.0:1789760736.503008:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760736 -> 1789760737 00010000:00010000:3.0:1789760736.503020:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55068 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.503034:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55068 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.503043:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55068 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.503050:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760736.503054:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55068 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.503063:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55068 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.503083:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55068 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.503095:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55068 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20574208->20578303] (req 20574208->20578303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.503238:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508323584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24788us (25144us total) trans 4294972343 rc 0/0 00000100:00100000:3.0:1789760736.503248:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5055 00000100:00100000:3.0:1789760736.503259:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715553238015 time 715583142974 state 0x2 ref 0 00000001:00080000:2.0:1789760736.505874:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972343 is committed 00000100:00100000:2.0:1789760736.507213:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760736.507375:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508323712 00000100:00100000:1.0:1789760736.507383:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508323712 00000100:00100000:1.0:1789760736.507430:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715553238015 time 715583142974 state 0x0 ref 1 00000100:00100000:1.0:1789760736.507444:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715583403390 time 715583142974 state 0x1 ref 1 00000100:00100000:0.0:1789760736.507633:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715583403390 time 715612341363 state 0x0 ref 1 00000100:00100000:0.0:1789760736.507646:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508323712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760736.507690:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20578304, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:0.0:1789760736.507699:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5506f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.507715:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760736.507720:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.507726:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5506f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.507739:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5506f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.541474:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972344 is committed 00002000:00010000:2.0:1789760736.542894:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20578304 -> 20582400 00010000:00010000:2.0:1789760736.542912:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5506f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.542932:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5506f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.542941:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5506f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.542949:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760736.542953:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5506f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.542963:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5506f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.542982:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5506f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.542991:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5506f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20578304->20582399] (req 20578304->20582399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.543094:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508323712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35446us (35882us total) trans 4294972344 rc 0/0 00000100:00100000:2.0:1789760736.543105:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5056 00000100:00100000:2.0:1789760736.543124:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715583403390 time 715612341363 state 0x2 ref 0 00000100:00100000:0.0:1789760736.546674:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760736.546760:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508323840 00000100:00100000:3.0:1789760736.546774:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508323840 00000100:00100000:3.0:1789760736.546828:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715583403390 time 715612341363 state 0x0 ref 1 00000100:00100000:3.0:1789760736.546844:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715612601779 time 715612341363 state 0x1 ref 1 00000100:00100000:3.0:1789760736.546871:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715612601779 time 715651582579 state 0x0 ref 1 00000100:00100000:3.0:1789760736.546884:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508323840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760736.546931:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20582400, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:3.0:1789760736.546940:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55076 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.546956:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.546960:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.546967:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55076 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.546985:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55076 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.583886:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972345 is committed 00002000:00010000:2.0:1789760736.584126:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20582400 -> 20586496 00010000:00010000:2.0:1789760736.584142:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55076 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.584154:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55076 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.584425:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55076 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.584436:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760736.584441:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55076 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.584451:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55076 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.584538:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55076 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.584546:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55076 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20582400->20586495] (req 20582400->20586495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.584789:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508323840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37902us (38119us total) trans 4294972345 rc 0/0 00000100:00100000:2.0:1789760736.584800:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5057 00000100:00100000:2.0:1789760736.584810:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715612601779 time 715651582579 state 0x2 ref 0 00000100:00100000:1.0:1789760736.590686:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760736.590792:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508323968 00000100:00100000:3.0:1789760736.590856:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508323968 00000100:00100000:3.0:1789760736.590913:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715612601779 time 715651582579 state 0x0 ref 1 00000100:00100000:3.0:1789760736.590929:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715651842995 time 715651582579 state 0x1 ref 1 00000100:00100000:3.0:1789760736.590959:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715651842995 time 715695669838 state 0x0 ref 1 00000100:00100000:3.0:1789760736.590973:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508323968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760736.591200:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20586496, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:3.0:1789760736.591211:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5507d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.591227:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.591231:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.591238:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5507d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.591253:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5507d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.626924:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972346 is committed 00002000:00010000:0.0:1789760736.627093:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20586496 -> 20590592 00010000:00010000:0.0:1789760736.627117:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5507d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.627131:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5507d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.627140:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5507d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.627148:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760736.627152:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5507d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.627161:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5507d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.627183:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5507d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.627192:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5507d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20586496->20590591] (req 20586496->20590591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.627358:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508323968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36382us (36676us total) trans 4294972346 rc 0/0 00000100:00100000:0.0:1789760736.627369:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5058 00000100:00100000:0.0:1789760736.627378:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715651842995 time 715695669838 state 0x2 ref 0 00000100:00100000:1.0:1789760736.631621:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760736.631906:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508324096 00000100:00100000:2.0:1789760736.631916:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508324096 00000100:00100000:2.0:1789760736.631968:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715651842995 time 715695669838 state 0x0 ref 1 00000100:00100000:2.0:1789760736.631982:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715695930254 time 715695669838 state 0x1 ref 1 00000100:00100000:2.0:1789760736.632005:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715695930254 time 715736716658 state 0x0 ref 1 00000100:00100000:2.0:1789760736.632017:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508324096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760736.632062:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20590592, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:2.0:1789760736.632072:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55084 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.632087:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760736.632091:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.632096:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55084 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.632228:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55084 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.663122:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972347 is committed 00002000:00010000:2.0:1789760736.663188:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20590592 -> 20594688 00010000:00010000:2.0:1789760736.663203:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55084 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.663221:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55084 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.663231:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55084 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.663238:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760736.663243:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55084 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.663252:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55084 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.663274:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55084 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.663283:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55084 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20590592->20594687] (req 20590592->20594687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.663410:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508324096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31389us (31845us total) trans 4294972347 rc 0/0 00000100:00100000:2.0:1789760736.663421:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5059 00000100:00100000:2.0:1789760736.663431:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715695930254 time 715736716658 state 0x2 ref 0 00000100:00100000:2.0:1789760736.667836:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760736.667941:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508324224 00000100:00100000:3.0:1789760736.667950:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508324224 00000100:00100000:3.0:1789760736.668001:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715695930254 time 715736716658 state 0x0 ref 1 00000100:00100000:3.0:1789760736.668015:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715736977074 time 715736716658 state 0x1 ref 1 00000100:00100000:3.0:1789760736.668125:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715736977074 time 715772833853 state 0x0 ref 1 00000100:00100000:3.0:1789760736.668138:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508324224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760736.668184:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20594688, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:3.0:1789760736.668194:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5508b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.668218:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.668223:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.668229:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5508b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.668243:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5508b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.703640:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972348 is committed 00002000:00010000:3.0:1789760736.703846:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20594688 -> 20598784 00010000:00010000:3.0:1789760736.703862:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5508b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.703875:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5508b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.703889:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5508b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.703897:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760736.703902:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5508b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.703911:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5508b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.703931:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5508b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.703939:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5508b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20594688->20598783] (req 20594688->20598783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.704035:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508324224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35894us (36202us total) trans 4294972348 rc 0/0 00000100:00100000:3.0:1789760736.704045:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5060 00000100:00100000:3.0:1789760736.704055:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715736977074 time 715772833853 state 0x2 ref 0 00000100:00100000:3.0:1789760736.711961:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760736.712058:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508324352 00000100:00100000:0.0:1789760736.712068:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508324352 00000100:00100000:0.0:1789760736.712127:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715736977074 time 715772833853 state 0x0 ref 1 00000100:00100000:0.0:1789760736.712141:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715773094269 time 715772833853 state 0x1 ref 1 00000100:00100000:0.0:1789760736.712168:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715773094269 time 715816879198 state 0x0 ref 1 00000100:00100000:0.0:1789760736.712180:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508324352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760736.712226:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20598784, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:0.0:1789760736.712235:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55092 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.712250:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760736.712255:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.712260:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55092 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.712279:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55092 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760736.748451:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20598784 -> 20602880 00010000:00010000:3.0:1789760736.748480:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55092 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.748500:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55092 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.748508:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55092 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.748514:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760736.748519:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55092 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.748529:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55092 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.748549:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55092 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.748557:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55092 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20598784->20602879] (req 20598784->20602879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.748656:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508324352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36472us (36699us total) trans 4294972349 rc 0/0 00000100:00100000:3.0:1789760736.748667:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5061 00000100:00100000:3.0:1789760736.748677:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715773094269 time 715816879198 state 0x2 ref 0 00000100:00100000:3.0:1789760736.752652:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760736.753254:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508324480 00000100:00100000:0.0:1789760736.753265:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508324480 00000100:00100000:0.0:1789760736.753352:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715773094269 time 715816879198 state 0x0 ref 1 00000100:00100000:0.0:1789760736.753367:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715817139614 time 715816879198 state 0x1 ref 1 00000100:00100000:0.0:1789760736.753397:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715817139614 time 715858106973 state 0x0 ref 1 00000100:00100000:0.0:1789760736.753409:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508324480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760736.753469:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20602880, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:0.0:1789760736.753478:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55099 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.753493:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760736.753497:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.753504:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55099 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.753518:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55099 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760736.754474:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972349 is committed 00002000:00010000:2.0:1789760736.790840:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20602880 -> 20606976 00010000:00010000:2.0:1789760736.790860:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55099 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.790874:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55099 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.790884:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55099 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.790891:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760736.790896:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55099 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.790904:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55099 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.790923:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55099 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.790931:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55099 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20602880->20606975] (req 20602880->20606975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760736.794619:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760736.794659:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508324608 00000100:00100000:3.0:1789760736.794668:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508324608 00000100:00100000:3.0:1789760736.794726:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715858367389 time 715858106973 state 0x1 ref 2 00000100:00100000:3.0:1789760736.794750:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715858367389 time 715899460682 state 0x0 ref 2 00000100:00100000:3.0:1789760736.794763:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c32ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2409:x1876699508324608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760736.794810:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20606976, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:3.0:1789760736.794819:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe550a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.794834:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760736.794838:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760736.794844:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe550a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760736.794857:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe550a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.795211:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2409:x1876699508324480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37615us (38379us total) trans 4294972350 rc 0/0 00000100:00100000:2.0:1789760736.795224:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5062 00000001:00080000:1.0:1789760736.795341:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972350 is committed 00000001:00080000:3.0:1789760736.846962:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972351 is committed 00002000:00010000:2.0:1789760736.847032:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20606976 -> 20611072 00010000:00010000:2.0:1789760736.847049:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe550a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.847061:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe550a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.847070:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe550a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.847078:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760736.847083:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe550a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.847093:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe550a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.847126:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe550a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.847134:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe550a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20606976->20611071] (req 20606976->20611071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.847226:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c32ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508324608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52459us (52611us total) trans 4294972351 rc 0/0 00000100:00100000:2.0:1789760736.847239:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5063 00000100:00100000:2.0:1789760736.847249:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715858367389 time 715899460682 state 0x2 ref 0 00000100:00100000:3.0:1789760736.853119:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760736.853264:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508324736 00000100:00100000:0.0:1789760736.853274:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508324736 00000100:00100000:0.0:1789760736.853382:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715858367389 time 715899460682 state 0x0 ref 1 00000100:00100000:0.0:1789760736.853398:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715899721098 time 715899460682 state 0x1 ref 1 00000100:00100000:0.0:1789760736.853429:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715899721098 time 715958139914 state 0x0 ref 1 00000100:00100000:0.0:1789760736.853443:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c32f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508324736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760736.853493:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20611072, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:0.0:1789760736.853503:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.853517:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760736.853523:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760736.853528:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.853543:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760736.905647:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972352 is committed 00002000:00010000:2.0:1789760736.905910:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20611072 -> 20615168 00010000:00010000:2.0:1789760736.905926:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.905938:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.905947:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.905954:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760736.905958:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.905967:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.905988:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.905996:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20611072->20615167] (req 20611072->20615167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.906214:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c32f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508324736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52768us (53110us total) trans 4294972352 rc 0/0 00000100:00100000:2.0:1789760736.906225:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5064 00000100:00100000:2.0:1789760736.906235:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715899721098 time 715958139914 state 0x2 ref 0 00000100:00100000:3.0:1789760736.915897:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760736.916083:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508324864 00000100:00100000:2.0:1789760736.916093:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508324864 00000100:00100000:2.0:1789760736.916220:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715899721098 time 715958139914 state 0x0 ref 1 00000100:00100000:2.0:1789760736.916236:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715958400330 time 715958139914 state 0x1 ref 1 00000100:00100000:2.0:1789760736.916264:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715958400330 time 716020975409 state 0x0 ref 1 00000100:00100000:2.0:1789760736.916281:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508324864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760736.916413:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20615168, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:2.0:1789760736.916422:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe550ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.916439:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760736.916444:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.916450:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe550ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.916463:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe550ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760736.969053:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972353 is committed 00002000:00010000:0.0:1789760736.969214:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20615168 -> 20619264 00010000:00010000:0.0:1789760736.969229:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe550ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.969243:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe550ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.969260:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe550ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.969268:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760736.969274:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe550ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.969283:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe550ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.969366:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe550ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760736.969374:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe550ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20615168->20619263] (req 20615168->20619263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760736.969561:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508324864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53277us (53668us total) trans 4294972353 rc 0/0 00000100:00100000:0.0:1789760736.969572:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5065 00000100:00100000:0.0:1789760736.969583:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715958400330 time 716020975409 state 0x2 ref 0 00000100:00100000:3.0:1789760736.972718:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760736.972891:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508324992 00000100:00100000:2.0:1789760736.972901:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508324992 00000100:00100000:2.0:1789760736.972955:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 715958400330 time 716020975409 state 0x0 ref 1 00000100:00100000:2.0:1789760736.972970:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716021235825 time 716020975409 state 0x1 ref 1 00000100:00100000:2.0:1789760736.972998:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716021235825 time 716077709102 state 0x0 ref 1 00000100:00100000:2.0:1789760736.973010:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508324992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760736.973067:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20619264, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:2.0:1789760736.973075:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.973090:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760736.973095:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760736.973100:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.973122:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760736.995474:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20619264 -> 20623360 00010000:00010000:2.0:1789760736.995496:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.995515:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.995523:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.995530:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760736.995535:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.995544:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.995565:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760736.995572:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20619264->20623359] (req 20619264->20623359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760736.995673:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508324992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22660us (22960us total) trans 4294972354 rc 0/0 00000100:00100000:2.0:1789760736.995683:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5066 00000100:00100000:2.0:1789760736.995693:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716021235825 time 716077709102 state 0x2 ref 0 00000001:00080000:0.0:1789760736.997917:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972354 is committed 00000100:00100000:3.0:1789760737.000070:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760737.000497:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508325120 00000100:00100000:2.0:1789760737.000508:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508325120 00000100:00100000:2.0:1789760737.000573:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716021235825 time 716077709102 state 0x0 ref 1 00000100:00100000:2.0:1789760737.000588:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716077969518 time 716077709102 state 0x1 ref 1 00000100:00100000:2.0:1789760737.000613:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716077969518 time 716105323961 state 0x0 ref 1 00000100:00100000:2.0:1789760737.000625:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508325120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760737.000672:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20623360, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:2.0:1789760737.000681:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe550bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.000696:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760737.000701:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.000707:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe550bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.000721:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe550bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760737.002507:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789760737.022469:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760737.029746:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972355 is committed 00002000:00010000:2.0:1789760737.029852:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20623360 -> 20627456 00010000:00010000:2.0:1789760737.029867:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe550bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.029881:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe550bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.029895:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe550bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.029902:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760737.029907:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe550bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.029916:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe550bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.029936:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe550bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.029943:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe550bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20623360->20627455] (req 20623360->20627455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.030159:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508325120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29530us (30094us total) trans 4294972355 rc 0/0 00000100:00100000:2.0:1789760737.030169:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5067 00000100:00100000:2.0:1789760737.030178:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716077969518 time 716105323961 state 0x2 ref 0 00000100:00100000:0.0:1789760737.033658:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760737.033733:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508325248 00000100:00100000:2.0:1789760737.033742:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508325248 00000100:00100000:2.0:1789760737.033791:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716077969518 time 716105323961 state 0x0 ref 1 00000100:00100000:2.0:1789760737.033805:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716105584377 time 716105323961 state 0x1 ref 1 00000100:00100000:2.0:1789760737.033860:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716105584377 time 716138570462 state 0x0 ref 1 00000100:00100000:2.0:1789760737.033871:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508325248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760737.033916:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20627456, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:2.0:1789760737.033925:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.033940:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760737.033944:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.033950:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.033964:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.063617:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972356 is committed 00002000:00010000:2.0:1789760737.063734:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20627456 -> 20631552 00010000:00010000:2.0:1789760737.063750:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.063763:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.063772:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.063779:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760737.063785:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.063794:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.063814:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.063822:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20627456->20631551] (req 20627456->20631551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.063994:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508325248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30118us (30338us total) trans 4294972356 rc 0/0 00000100:00100000:2.0:1789760737.064005:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5068 00000100:00100000:2.0:1789760737.064015:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716105584377 time 716138570462 state 0x2 ref 0 00000100:00100000:0.0:1789760737.068136:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760737.068253:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508325376 00000100:00100000:2.0:1789760737.068264:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508325376 00000100:00100000:2.0:1789760737.070366:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716105584377 time 716138570462 state 0x0 ref 1 00000100:00100000:2.0:1789760737.070387:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716138830878 time 716138570462 state 0x1 ref 1 00000100:00100000:3.0:1789760737.070545:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716138830878 time 716175253793 state 0x0 ref 1 00000100:00100000:3.0:1789760737.070568:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508325376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760737.070615:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20631552, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:3.0:1789760737.070624:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.070640:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760737.070645:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.070651:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.070665:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.130394:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972357 is committed 00002000:00010000:3.0:1789760737.130626:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20631552 -> 20635648 00010000:00010000:3.0:1789760737.130641:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.130653:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.130662:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.130674:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760737.130679:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.130687:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.130707:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.130715:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20631552->20635647] (req 20631552->20635647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.130813:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508325376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60251us (62681us total) trans 4294972357 rc 0/0 00000100:00100000:3.0:1789760737.130823:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5069 00000100:00100000:3.0:1789760737.130832:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716138830878 time 716175253793 state 0x2 ref 0 00000100:00100000:1.0:1789760737.136750:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760737.136840:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508325504 00000100:00100000:0.0:1789760737.136850:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508325504 00000100:00100000:0.0:1789760737.136902:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716138830878 time 716175253793 state 0x0 ref 1 00000100:00100000:0.0:1789760737.136918:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716175514209 time 716175253793 state 0x1 ref 1 00000100:00100000:0.0:1789760737.136982:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716175514209 time 716241691955 state 0x0 ref 1 00000100:00100000:0.0:1789760737.136995:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508325504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760737.137056:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20635648, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:0.0:1789760737.137065:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.137081:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760737.137085:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.137092:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.137115:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.154434:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760737.154469:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049891fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534068608:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760737.154537:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760737.154799:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534068608 00000100:00100000:1.0:1789760737.154807:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534068608 00000100:00100000:1.0:1789760737.154863:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 428 00000100:00100000:1.0:1789760737.154873:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534068608:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760737.154910:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760737.155004:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534068608:12345-0@lo:41:osp-pre-0-1.0 Request processed in 132us (465us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760737.155011:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 428 00000100:00100000:2.0:1789760737.155270:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049891fd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534068608:0@lo:41:osp-pre-0-1.0 00000001:00080000:0.0:1789760737.212715:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972358 is committed 00002000:00010000:3.0:1789760737.212842:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20635648 -> 20639744 00010000:00010000:3.0:1789760737.212858:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.212869:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.212883:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.212890:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760737.212895:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.212904:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.212925:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.212933:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe550d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20635648->20639743] (req 20635648->20639743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.213046:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508325504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 76048us (76300us total) trans 4294972358 rc 0/0 00000100:00100000:3.0:1789760737.213057:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5070 00000100:00100000:3.0:1789760737.213067:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716175514209 time 716241691955 state 0x2 ref 0 00000100:00100000:0.0:1789760737.228210:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760737.229159:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508325632 00000100:00100000:2.0:1789760737.229172:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508325632 00000100:00100000:2.0:1789760737.229231:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716175514209 time 716241691955 state 0x0 ref 1 00000100:00100000:2.0:1789760737.229247:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716241952371 time 716241691955 state 0x1 ref 1 00000100:00100000:2.0:1789760737.229282:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716241952371 time 716333992438 state 0x0 ref 1 00000100:00100000:2.0:1789760737.229334:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508325632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760737.229395:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20639744, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:2.0:1789760737.229404:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.229421:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760737.229426:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.229433:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.229447:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.281887:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972359 is committed 00002000:00010000:0.0:1789760737.282027:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20639744 -> 20643840 00010000:00010000:0.0:1789760737.282045:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.282066:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.282077:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.282085:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760737.282091:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.282101:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.282133:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.282140:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe550d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20639744->20643839] (req 20639744->20643839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.282241:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508325632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52905us (54046us total) trans 4294972359 rc 0/0 00000100:00100000:0.0:1789760737.282250:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5071 00000100:00100000:0.0:1789760737.282259:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716241952371 time 716333992438 state 0x2 ref 0 00000100:00100000:0.0:1789760737.294526:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760737.294611:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508325760 00000100:00100000:3.0:1789760737.294622:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508325760 00000100:00100000:3.0:1789760737.294678:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716241952371 time 716333992438 state 0x0 ref 1 00000100:00100000:3.0:1789760737.294698:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716334252854 time 716333992438 state 0x1 ref 1 00000100:00100000:3.0:1789760737.294729:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716334252854 time 716399440291 state 0x0 ref 1 00000100:00100000:3.0:1789760737.294742:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508325760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760737.294793:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20643840, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:3.0:1789760737.294802:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe550df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.294816:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760737.294820:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.294827:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe550df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.294841:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe550df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760737.329161:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972360 is committed 00002000:00010000:0.0:1789760737.329335:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20643840 -> 20647936 00010000:00010000:0.0:1789760737.329350:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe550df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.329363:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe550df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.329373:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe550df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.329381:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760737.329385:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe550df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.329394:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe550df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.329424:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe550df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.329431:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe550df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20643840->20647935] (req 20643840->20647935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.329524:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508325760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34779us (35003us total) trans 4294972360 rc 0/0 00000100:00100000:0.0:1789760737.329650:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5072 00000100:00100000:0.0:1789760737.329661:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716334252854 time 716399440291 state 0x2 ref 0 00000100:00100000:3.0:1789760737.332970:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760737.333328:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508325888 00000100:00100000:0.0:1789760737.333339:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508325888 00000100:00100000:0.0:1789760737.333393:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716334252854 time 716399440291 state 0x0 ref 1 00000100:00100000:0.0:1789760737.333420:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716399700707 time 716399440291 state 0x1 ref 1 00000100:00100000:0.0:1789760737.333448:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716399700707 time 716438158474 state 0x0 ref 1 00000100:00100000:0.0:1789760737.333461:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508325888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760737.333506:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20647936, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:0.0:1789760737.333515:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe550e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.333531:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760737.333546:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.333552:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe550e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.333565:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe550e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.383073:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972361 is committed 00002000:00010000:2.0:1789760737.383338:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20647936 -> 20652032 00010000:00010000:2.0:1789760737.383353:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe550e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.383366:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe550e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.383376:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe550e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.383383:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760737.383388:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe550e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.383395:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe550e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.383413:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe550e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.383421:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe550e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20647936->20652031] (req 20647936->20652031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.383508:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508325888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50045us (50543us total) trans 4294972361 rc 0/0 00000100:00100000:2.0:1789760737.383519:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5073 00000100:00100000:2.0:1789760737.383528:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716399700707 time 716438158474 state 0x2 ref 0 00000100:00100000:3.0:1789760737.387947:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760737.388119:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508326016 00000100:00100000:0.0:1789760737.388129:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508326016 00000100:00100000:0.0:1789760737.388194:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716399700707 time 716438158474 state 0x0 ref 1 00000100:00100000:0.0:1789760737.388209:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716438418890 time 716438158474 state 0x1 ref 1 00000100:00100000:0.0:1789760737.388242:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716438418890 time 716492952235 state 0x0 ref 1 00000100:00100000:0.0:1789760737.388255:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508326016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760737.388350:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20652032, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:0.0:1789760737.388359:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe550ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.388374:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760737.388378:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.388385:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe550ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.388399:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe550ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760737.464938:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20652032 -> 20656128 00010000:00010000:2.0:1789760737.464957:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe550ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.464975:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe550ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.464983:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe550ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.464990:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760737.465005:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe550ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.465014:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe550ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.465033:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe550ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.465040:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe550ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20652032->20656127] (req 20652032->20656127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.465141:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508326016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 76883us (77198us total) trans 4294972362 rc 0/0 00000100:00100000:2.0:1789760737.465152:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5074 00000100:00100000:2.0:1789760737.465162:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716438418890 time 716492952235 state 0x2 ref 0 00000001:00080000:1.0:1789760737.465563:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972362 is committed 00000100:00100000:0.0:1789760737.469752:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760737.469864:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508326144 00000100:00100000:3.0:1789760737.469874:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508326144 00000100:00100000:3.0:1789760737.469932:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716438418890 time 716492952235 state 0x0 ref 1 00000100:00100000:3.0:1789760737.469949:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716493212651 time 716492952235 state 0x1 ref 1 00000100:00100000:3.0:1789760737.469978:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716493212651 time 716574688432 state 0x0 ref 1 00000100:00100000:3.0:1789760737.469992:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508326144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760737.470041:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20656128, mtime: 0x6aad94e1, atime: 0x0, ctime: 0x6aad94e1, blocks: 0xa007 00010000:00010000:3.0:1789760737.470049:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.470064:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760737.470068:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.470074:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.470089:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760737.518350:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20656128 -> 20660224 00002000:00010000:3.0:1789760737.518358:0:14638:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760737 -> 1789760738 00002000:00010000:3.0:1789760737.518361:0:14638:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760737 -> 1789760738 00010000:00010000:3.0:1789760737.518374:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.518386:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.518395:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.518407:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760737.518412:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.518421:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.518440:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.518448:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe550f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20656128->20660223] (req 20656128->20660223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.518552:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508326144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48557us (48804us total) trans 4294972363 rc 0/0 00000100:00100000:3.0:1789760737.518562:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5075 00000100:00100000:3.0:1789760737.518572:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716493212651 time 716574688432 state 0x2 ref 0 00000001:00080000:1.0:1789760737.518864:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972363 is committed 00000100:00100000:3.0:1789760737.523492:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760737.523662:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508326272 00000100:00100000:2.0:1789760737.523673:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508326272 00000100:00100000:0.0:1789760737.523858:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716493212651 time 716574688432 state 0x0 ref 1 00000100:00100000:0.0:1789760737.523873:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716574948848 time 716574688432 state 0x1 ref 1 00000100:00100000:0.0:1789760737.523938:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716574948848 time 716628647787 state 0x0 ref 1 00000100:00100000:0.0:1789760737.523950:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508326272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760737.523998:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20660224, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:0.0:1789760737.524007:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe550fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.524021:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760737.524025:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.524031:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe550fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.524044:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe550fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760737.574692:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20660224 -> 20664320 00010000:00010000:2.0:1789760737.574811:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe550fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.574824:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe550fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.574833:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe550fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.574840:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760737.574845:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe550fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.574855:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe550fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.574874:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe550fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.574881:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe550fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20660224->20664319] (req 20660224->20664319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.574972:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508326272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51019us (51486us total) trans 4294972364 rc 0/0 00000100:00100000:2.0:1789760737.574983:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5076 00000100:00100000:2.0:1789760737.574992:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716574948848 time 716628647787 state 0x2 ref 0 00000001:00080000:3.0:1789760737.575587:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972364 is committed 00000100:00100000:2.0:1789760737.579398:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760737.579617:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508326400 00000100:00100000:0.0:1789760737.579626:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508326400 00000100:00100000:0.0:1789760737.579676:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716574948848 time 716628647787 state 0x0 ref 1 00000100:00100000:0.0:1789760737.579690:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716628908203 time 716628647787 state 0x1 ref 1 00000100:00100000:0.0:1789760737.579717:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716628908203 time 716684427825 state 0x0 ref 1 00000100:00100000:0.0:1789760737.579728:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508326400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760737.579773:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20664320, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:0.0:1789760737.579782:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55102 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.579797:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760737.579801:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760737.579808:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55102 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.579828:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55102 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.605767:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972365 is committed 00002000:00010000:2.0:1789760737.606692:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20664320 -> 20668416 00010000:00010000:2.0:1789760737.606709:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55102 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.606722:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55102 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.606731:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55102 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.606738:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760737.606743:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55102 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.606752:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55102 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.606773:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55102 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.606782:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55102 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20664320->20668415] (req 20664320->20668415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.606885:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508326400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27153us (27490us total) trans 4294972365 rc 0/0 00000100:00100000:2.0:1789760737.606899:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5077 00000100:00100000:2.0:1789760737.606910:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716628908203 time 716684427825 state 0x2 ref 0 00000100:00100000:0.0:1789760737.613125:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760737.613504:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508326528 00000100:00100000:3.0:1789760737.613515:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508326528 00000100:00100000:3.0:1789760737.613572:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716628908203 time 716684427825 state 0x0 ref 1 00000100:00100000:3.0:1789760737.613587:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716684688241 time 716684427825 state 0x1 ref 1 00000100:00100000:3.0:1789760737.613626:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716684688241 time 716718336093 state 0x0 ref 1 00000100:00100000:3.0:1789760737.613639:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508326528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760737.613683:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20668416, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:3.0:1789760737.613692:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55109 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.613708:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760737.613712:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.613718:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55109 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.613731:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55109 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760737.664646:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972366 is committed 00002000:00010000:2.0:1789760737.664792:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20668416 -> 20672512 00010000:00010000:2.0:1789760737.664807:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55109 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.664821:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55109 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.664830:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55109 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.664837:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760737.664842:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55109 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.664852:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55109 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.664872:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55109 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.664880:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55109 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20668416->20672511] (req 20668416->20672511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.664982:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508326528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51340us (51873us total) trans 4294972366 rc 0/0 00000100:00100000:2.0:1789760737.664993:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5078 00000100:00100000:2.0:1789760737.665003:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716684688241 time 716718336093 state 0x2 ref 0 00000100:00100000:1.0:1789760737.670141:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760737.670250:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508326656 00000100:00100000:2.0:1789760737.670259:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508326656 00000100:00100000:2.0:1789760737.670427:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716684688241 time 716718336093 state 0x0 ref 1 00000100:00100000:2.0:1789760737.670443:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716718596509 time 716718336093 state 0x1 ref 1 00000100:00100000:2.0:1789760737.670625:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716718596509 time 716775333753 state 0x0 ref 1 00000100:00100000:2.0:1789760737.670638:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508326656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760737.670688:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20672512, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760737.670697:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55110 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.670715:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760737.670720:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.670726:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55110 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.670740:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55110 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760737.702399:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972367 is committed 00002000:00010000:2.0:1789760737.702717:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20672512 -> 20676608 00010000:00010000:2.0:1789760737.702734:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55110 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.702748:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55110 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.702757:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55110 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.702764:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760737.702769:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55110 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.702779:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55110 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.702800:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55110 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.702808:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55110 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20672512->20676607] (req 20672512->20676607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.702906:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508326656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32264us (32888us total) trans 4294972367 rc 0/0 00000100:00100000:2.0:1789760737.702919:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5079 00000100:00100000:2.0:1789760737.702929:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716718596509 time 716775333753 state 0x2 ref 0 00000100:00100000:0.0:1789760737.708542:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760737.708623:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508326784 00000100:00100000:2.0:1789760737.708633:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508326784 00000100:00100000:2.0:1789760737.708688:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716718596509 time 716775333753 state 0x0 ref 1 00000100:00100000:2.0:1789760737.708703:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716775594169 time 716775333753 state 0x1 ref 1 00000100:00100000:2.0:1789760737.708735:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716775594169 time 716813445761 state 0x0 ref 1 00000100:00100000:2.0:1789760737.708749:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508326784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760737.708799:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20676608, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760737.708808:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55117 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.708823:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760737.708828:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.708833:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55117 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.708848:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55117 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.747244:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972368 is committed 00002000:00010000:3.0:1789760737.747411:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20676608 -> 20680704 00010000:00010000:3.0:1789760737.747426:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55117 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.747439:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55117 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.747447:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55117 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.747534:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760737.747539:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55117 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.747547:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55117 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.747566:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55117 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.747574:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55117 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20676608->20680703] (req 20676608->20680703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760737.747790:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508326784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39038us (39328us total) trans 4294972368 rc 0/0 00000100:00100000:3.0:1789760737.747800:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5080 00000100:00100000:3.0:1789760737.747810:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716775594169 time 716813445761 state 0x2 ref 0 00000100:00100000:0.0:1789760737.751768:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760737.751885:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508326912 00000100:00100000:2.0:1789760737.751896:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508326912 00000100:00100000:2.0:1789760737.751951:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716775594169 time 716813445761 state 0x0 ref 1 00000100:00100000:2.0:1789760737.751966:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716813706177 time 716813445761 state 0x1 ref 1 00000100:00100000:2.0:1789760737.752224:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716813706177 time 716856932401 state 0x0 ref 1 00000100:00100000:2.0:1789760737.752238:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508326912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760737.752359:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20680704, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760737.752368:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5511e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.752385:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760737.752389:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.752396:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5511e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.752408:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5511e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760737.787091:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20680704 -> 20684800 00010000:00010000:2.0:1789760737.787121:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5511e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.787134:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5511e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.787143:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5511e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.787160:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760737.787166:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5511e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.787176:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5511e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.787194:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5511e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.787202:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5511e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20680704->20684799] (req 20680704->20684799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.787330:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508326912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35090us (35569us total) trans 4294972369 rc 0/0 00000100:00100000:2.0:1789760737.787341:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5081 00000100:00100000:2.0:1789760737.787351:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716813706177 time 716856932401 state 0x2 ref 0 00000001:00080000:1.0:1789760737.788013:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972369 is committed 00000100:00100000:1.0:1789760737.790759:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760737.790973:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508327040 00000100:00100000:2.0:1789760737.790982:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508327040 00000100:00100000:2.0:1789760737.792114:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716813706177 time 716856932401 state 0x0 ref 1 00000100:00100000:2.0:1789760737.792131:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716857192817 time 716856932401 state 0x1 ref 1 00000100:00100000:2.0:1789760737.793169:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716857192817 time 716897874236 state 0x0 ref 1 00000100:00100000:2.0:1789760737.793195:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508327040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760737.793249:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20684800, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760737.793258:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55125 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.793274:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760737.793297:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.793304:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55125 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.793319:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55125 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.820343:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972370 is committed 00002000:00010000:2.0:1789760737.820974:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20684800 -> 20688896 00010000:00010000:2.0:1789760737.820992:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55125 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.821005:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55125 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.821016:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55125 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.821086:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760737.821092:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55125 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.821112:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55125 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.821132:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55125 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.821141:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55125 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20684800->20688895] (req 20684800->20688895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.821418:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508327040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28227us (30659us total) trans 4294972370 rc 0/0 00000100:00100000:2.0:1789760737.821428:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5082 00000100:00100000:2.0:1789760737.821437:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716857192817 time 716897874236 state 0x2 ref 0 00000100:00100000:0.0:1789760737.825637:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760737.825703:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508327168 00000100:00100000:3.0:1789760737.825713:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508327168 00000100:00100000:3.0:1789760737.825769:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716857192817 time 716897874236 state 0x0 ref 1 00000100:00100000:3.0:1789760737.825785:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716898134652 time 716897874236 state 0x1 ref 1 00000100:00100000:3.0:1789760737.825816:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716898134652 time 716930526320 state 0x0 ref 1 00000100:00100000:3.0:1789760737.825831:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508327168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760737.825879:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20688896, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:3.0:1789760737.825889:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5512c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.825904:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760737.825908:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760737.825914:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5512c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760737.825928:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5512c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.863982:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972371 is committed 00002000:00010000:2.0:1789760737.864038:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20688896 -> 20692992 00010000:00010000:2.0:1789760737.864054:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5512c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.864067:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5512c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.864075:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5512c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.864082:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760737.864087:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5512c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.864096:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5512c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.864124:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5512c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.864132:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5512c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20688896->20692991] (req 20688896->20692991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760737.864235:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508327168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38400us (38603us total) trans 4294972371 rc 0/0 00000100:00100000:2.0:1789760737.864246:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5083 00000100:00100000:2.0:1789760737.864255:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716898134652 time 716930526320 state 0x2 ref 0 00000100:00100000:0.0:1789760737.869415:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760737.869650:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508327296 00000100:00100000:2.0:1789760737.869667:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508327296 00000100:00100000:2.0:1789760737.869728:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716898134652 time 716930526320 state 0x0 ref 1 00000100:00100000:2.0:1789760737.869744:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716930786736 time 716930526320 state 0x1 ref 1 00000100:00100000:2.0:1789760737.869780:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716930786736 time 716974489803 state 0x0 ref 1 00000100:00100000:2.0:1789760737.869796:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508327296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760737.869848:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20692992, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760737.869858:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55133 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.869874:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760737.870147:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.870155:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55133 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.870171:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55133 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760737.904958:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972372 is committed 00002000:00010000:0.0:1789760737.905089:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20692992 -> 20697088 00010000:00010000:0.0:1789760737.905118:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55133 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.905133:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55133 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.905143:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55133 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.905150:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760737.905155:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55133 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.905165:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55133 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.905192:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55133 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.905201:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55133 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20692992->20697087] (req 20692992->20697087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.905374:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508327296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35576us (35969us total) trans 4294972372 rc 0/0 00000100:00100000:0.0:1789760737.905388:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5084 00000100:00100000:0.0:1789760737.905400:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716930786736 time 716974489803 state 0x2 ref 0 00000100:00100000:1.0:1789760737.913403:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760737.913619:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508327424 00000100:00100000:2.0:1789760737.913639:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508327424 00000100:00100000:2.0:1789760737.913712:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716930786736 time 716974489803 state 0x0 ref 1 00000100:00100000:2.0:1789760737.913733:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716974750219 time 716974489803 state 0x1 ref 1 00000100:00100000:2.0:1789760737.913772:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716974750219 time 717018481798 state 0x0 ref 1 00000100:00100000:2.0:1789760737.913787:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508327424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760737.913862:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20697088, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760737.913874:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5513a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.913895:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760737.913900:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.913908:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5513a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.913926:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5513a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760737.963927:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20697088 -> 20701184 00010000:00010000:0.0:1789760737.963962:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5513a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.963979:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5513a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.963988:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5513a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.964005:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760737.964010:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5513a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.964024:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5513a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.964046:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5513a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760737.964188:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5513a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20697088->20701183] (req 20697088->20701183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760737.964414:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508327424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50623us (51014us total) trans 4294972373 rc 0/0 00000100:00100000:0.0:1789760737.964427:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5085 00000100:00100000:0.0:1789760737.964437:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716974750219 time 717018481798 state 0x2 ref 0 00000001:00080000:1.0:1789760737.970653:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972373 is committed 00000100:00100000:0.0:1789760737.976742:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760737.976880:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508327552 00000100:00100000:2.0:1789760737.976891:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508327552 00000100:00100000:2.0:1789760737.976954:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 716974750219 time 717018481798 state 0x0 ref 1 00000100:00100000:2.0:1789760737.976970:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717018742214 time 717018481798 state 0x1 ref 1 00000100:00100000:2.0:1789760737.977009:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717018742214 time 717081719432 state 0x0 ref 1 00000100:00100000:2.0:1789760737.977033:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508327552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760737.977082:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20701184, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760737.977091:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55141 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.977129:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760737.977134:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760737.977746:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55141 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760737.977844:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55141 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.027826:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972374 is committed 00002000:00010000:0.0:1789760738.027934:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20701184 -> 20705280 00010000:00010000:0.0:1789760738.027951:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55141 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.027964:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55141 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.027973:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55141 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.027986:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760738.027991:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55141 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.028001:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55141 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.028036:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55141 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.028044:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55141 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20701184->20705279] (req 20701184->20705279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.028163:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508327552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51137us (51426us total) trans 4294972374 rc 0/0 00000100:00100000:0.0:1789760738.028173:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5086 00000100:00100000:0.0:1789760738.028190:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717018742214 time 717081719432 state 0x2 ref 0 00000100:00100000:2.0:1789760738.033621:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760738.035617:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508327680 00000100:00100000:0.0:1789760738.035630:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508327680 00000100:00100000:2.0:1789760738.035759:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717018742214 time 717081719432 state 0x0 ref 1 00000100:00100000:2.0:1789760738.035777:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717081979848 time 717081719432 state 0x1 ref 1 00000100:00100000:2.0:1789760738.035812:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717081979848 time 717140522038 state 0x0 ref 1 00000100:00100000:2.0:1789760738.035825:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508327680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760738.035875:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20705280, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760738.035885:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55148 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.035903:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.035907:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.035914:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55148 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.035929:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55148 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.046725:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760738.068913:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972375 is committed 00002000:00010000:2.0:1789760738.069093:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20705280 -> 20709376 00010000:00010000:2.0:1789760738.069120:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55148 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.069135:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55148 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.069150:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55148 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.069158:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760738.069163:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55148 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.069173:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55148 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.071362:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55148 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.071377:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55148 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20705280->20709375] (req 20705280->20709375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.071501:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508327680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35670us (37883us total) trans 4294972375 rc 0/0 00000100:00100000:2.0:1789760738.071513:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5087 00000100:00100000:2.0:1789760738.071526:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717081979848 time 717140522038 state 0x2 ref 0 00000100:00100000:2.0:1789760738.077444:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760738.077549:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508327808 00000100:00100000:0.0:1789760738.077560:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508327808 00000100:00100000:0.0:1789760738.077620:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717081979848 time 717140522038 state 0x0 ref 1 00000100:00100000:0.0:1789760738.077641:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717140782454 time 717140522038 state 0x1 ref 1 00000100:00100000:0.0:1789760738.077668:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717140782454 time 717182378845 state 0x0 ref 1 00000100:00100000:0.0:1789760738.077681:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508327808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760738.077734:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20709376, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:0.0:1789760738.077744:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5514f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.077760:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760738.077764:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.077771:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5514f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.077785:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5514f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760738.125492:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972376 is committed 00002000:00010000:0.0:1789760738.125895:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20709376 -> 20713472 00010000:00010000:0.0:1789760738.125912:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5514f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.125926:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5514f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.125935:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5514f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.125947:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760738.125952:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5514f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.125961:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5514f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.125995:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5514f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.126004:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5514f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20709376->20713471] (req 20709376->20713471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.126114:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508327808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48418us (48663us total) trans 4294972376 rc 0/0 00000100:00100000:0.0:1789760738.126126:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5088 00000100:00100000:0.0:1789760738.126145:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717140782454 time 717182378845 state 0x2 ref 0 00000100:00100000:0.0:1789760738.130682:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760738.130772:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508327936 00000100:00100000:2.0:1789760738.130783:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508327936 00000100:00100000:2.0:1789760738.130843:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717140782454 time 717182378845 state 0x0 ref 1 00000100:00100000:2.0:1789760738.130859:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717182639261 time 717182378845 state 0x1 ref 1 00000100:00100000:2.0:1789760738.130890:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717182639261 time 717235600120 state 0x0 ref 1 00000100:00100000:2.0:1789760738.130903:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508327936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760738.130953:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20713472, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760738.130963:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55156 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.130983:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.130987:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.130995:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55156 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.131021:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55156 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760738.163156:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972377 is committed 00002000:00010000:0.0:1789760738.163264:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20713472 -> 20717568 00010000:00010000:0.0:1789760738.163280:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55156 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.163309:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55156 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.163324:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55156 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.163331:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760738.163336:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55156 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.163345:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55156 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.163366:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55156 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.163374:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55156 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20713472->20717567] (req 20713472->20717567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.163478:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508327936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32571us (32801us total) trans 4294972377 rc 0/0 00000100:00100000:0.0:1789760738.163488:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5089 00000100:00100000:0.0:1789760738.163498:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717182639261 time 717235600120 state 0x2 ref 0 00000100:00100000:0.0:1789760738.167139:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760738.167223:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508328064 00000100:00100000:3.0:1789760738.167234:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508328064 00000100:00100000:3.0:1789760738.167320:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717182639261 time 717235600120 state 0x0 ref 1 00000100:00100000:3.0:1789760738.167337:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717235860536 time 717235600120 state 0x1 ref 1 00000100:00100000:3.0:1789760738.167366:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717235860536 time 717272076382 state 0x0 ref 1 00000100:00100000:3.0:1789760738.167380:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508328064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760738.167432:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20717568, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:3.0:1789760738.167443:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5515d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.167473:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760738.167478:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.167486:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5515d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.167503:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5515d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.203278:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972378 is committed 00002000:00010000:3.0:1789760738.203516:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20717568 -> 20721664 00010000:00010000:3.0:1789760738.203533:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5515d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.203546:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5515d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.203555:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5515d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.203562:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760738.203567:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5515d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.203576:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5515d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.203600:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5515d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.203610:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5515d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20717568->20721663] (req 20717568->20721663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.203747:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508328064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36364us (36612us total) trans 4294972378 rc 0/0 00000100:00100000:3.0:1789760738.203758:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5090 00000100:00100000:3.0:1789760738.203769:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717235860536 time 717272076382 state 0x2 ref 0 00000100:00100000:2.0:1789760738.207730:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760738.207852:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508328192 00000100:00100000:0.0:1789760738.207862:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508328192 00000100:00100000:0.0:1789760738.207921:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717235860536 time 717272076382 state 0x0 ref 1 00000100:00100000:0.0:1789760738.207938:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717272336798 time 717272076382 state 0x1 ref 1 00000100:00100000:0.0:1789760738.208003:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717272336798 time 717312712688 state 0x0 ref 1 00000100:00100000:0.0:1789760738.208028:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508328192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760738.208082:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20721664, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:0.0:1789760738.208093:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55164 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.208122:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760738.208127:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.208134:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55164 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.208149:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55164 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.246960:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20721664 -> 20725760 00010000:00010000:2.0:1789760738.246981:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55164 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.246993:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55164 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.247006:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55164 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.247013:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760738.247018:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55164 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.247028:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55164 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.247051:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55164 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.247060:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55164 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20721664->20725759] (req 20721664->20725759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.247222:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508328192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39201us (39496us total) trans 4294972379 rc 0/0 00000100:00100000:2.0:1789760738.247233:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5091 00000100:00100000:2.0:1789760738.247243:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717272336798 time 717312712688 state 0x2 ref 0 00000001:00080000:3.0:1789760738.247455:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972379 is committed 00000100:00100000:0.0:1789760738.253812:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760738.253925:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508328320 00000100:00100000:2.0:1789760738.253935:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508328320 00000100:00100000:2.0:1789760738.253988:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717272336798 time 717312712688 state 0x0 ref 1 00000100:00100000:2.0:1789760738.254005:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717312973104 time 717312712688 state 0x1 ref 1 00000100:00100000:2.0:1789760738.254061:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717312973104 time 717358770938 state 0x0 ref 1 00000100:00100000:2.0:1789760738.254073:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508328320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760738.254138:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20725760, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760738.254148:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5516b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.254166:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.254170:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.254177:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5516b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.254192:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5516b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.291198:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20725760 -> 20729856 00010000:00010000:2.0:1789760738.291218:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5516b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.291232:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5516b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.291241:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5516b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.291278:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760738.291318:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5516b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.291329:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5516b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.291353:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5516b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.291360:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5516b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20725760->20729855] (req 20725760->20729855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.291562:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508328320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37484us (37753us total) trans 4294972380 rc 0/0 00000100:00100000:2.0:1789760738.291573:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5092 00000100:00100000:2.0:1789760738.291584:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717312973104 time 717358770938 state 0x2 ref 0 00000001:00080000:0.0:1789760738.292182:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972380 is committed 00000100:00100000:0.0:1789760738.298830:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760738.298916:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508328448 00000100:00100000:3.0:1789760738.298927:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508328448 00000100:00100000:3.0:1789760738.298985:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717312973104 time 717358770938 state 0x0 ref 1 00000100:00100000:3.0:1789760738.299001:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717359031354 time 717358770938 state 0x1 ref 1 00000100:00100000:3.0:1789760738.299033:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717359031354 time 717403743934 state 0x0 ref 1 00000100:00100000:3.0:1789760738.299046:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508328448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760738.299113:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20729856, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:3.0:1789760738.299124:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55172 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.299143:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760738.299147:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.299155:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55172 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.299176:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55172 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.334067:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972381 is committed 00002000:00010000:0.0:1789760738.334247:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20729856 -> 20733952 00010000:00010000:0.0:1789760738.334264:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55172 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.334277:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55172 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.334376:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55172 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.334384:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760738.334389:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55172 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.334400:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55172 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.334424:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55172 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.334433:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55172 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20729856->20733951] (req 20729856->20733951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.334593:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508328448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35542us (35767us total) trans 4294972381 rc 0/0 00000100:00100000:0.0:1789760738.334605:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5093 00000100:00100000:0.0:1789760738.334618:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717359031354 time 717403743934 state 0x2 ref 0 00000100:00100000:1.0:1789760738.341419:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760738.341515:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508328576 00000100:00100000:2.0:1789760738.341526:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508328576 00000100:00100000:2.0:1789760738.341590:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717359031354 time 717403743934 state 0x0 ref 1 00000100:00100000:2.0:1789760738.341607:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717404004350 time 717403743934 state 0x1 ref 1 00000100:00100000:2.0:1789760738.341635:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717404004350 time 717446345807 state 0x0 ref 1 00000100:00100000:2.0:1789760738.341649:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508328576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760738.341703:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20733952, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760738.341718:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55179 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.341736:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.341741:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.341749:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55179 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.341764:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55179 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.371719:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972382 is committed 00002000:00010000:0.0:1789760738.371799:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20733952 -> 20738048 00010000:00010000:0.0:1789760738.371815:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55179 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.371833:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55179 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.371841:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55179 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.371848:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760738.371853:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55179 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.371862:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55179 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.371886:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55179 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.371893:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55179 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20733952->20738047] (req 20733952->20738047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760738.372064:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508328576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30411us (30649us total) trans 4294972382 rc 0/0 00000100:00100000:0.0:1789760738.372074:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5094 00000100:00100000:0.0:1789760738.372085:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717404004350 time 717446345807 state 0x2 ref 0 00000100:00100000:1.0:1789760738.376065:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760738.376710:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508328704 00000100:00100000:2.0:1789760738.376722:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508328704 00000100:00100000:2.0:1789760738.376785:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717404004350 time 717446345807 state 0x0 ref 1 00000100:00100000:2.0:1789760738.376801:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717446606223 time 717446345807 state 0x1 ref 1 00000100:00100000:2.0:1789760738.376862:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717446606223 time 717481571499 state 0x0 ref 1 00000100:00100000:2.0:1789760738.376876:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508328704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760738.376927:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20738048, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760738.376937:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55180 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.376954:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.376959:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.376966:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55180 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.376982:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55180 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760738.408971:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20738048 -> 20742144 00010000:00010000:1.0:1789760738.408992:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55180 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.409005:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55180 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.409082:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55180 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.409090:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760738.409210:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55180 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.409223:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55180 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.409246:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55180 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760738.409254:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55180 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20738048->20742143] (req 20738048->20742143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760738.409450:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508328704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32562us (33381us total) trans 4294972383 rc 0/0 00000100:00100000:1.0:1789760738.409461:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5095 00000100:00100000:1.0:1789760738.409473:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717446606223 time 717481571499 state 0x2 ref 0 00000001:00080000:1.0:1789760738.409516:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972383 is committed 00000100:00100000:2.0:1789760738.414212:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760738.414271:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508328832 00000100:00100000:0.0:1789760738.414281:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508328832 00000100:00100000:0.0:1789760738.414368:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717446606223 time 717481571499 state 0x0 ref 1 00000100:00100000:0.0:1789760738.414385:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717481831915 time 717481571499 state 0x1 ref 1 00000100:00100000:3.0:1789760738.414500:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717481831915 time 717519206435 state 0x0 ref 1 00000100:00100000:3.0:1789760738.414516:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508328832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760738.414570:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20742144, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:3.0:1789760738.414579:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55187 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.414597:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760738.414602:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.414610:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55187 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.414636:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55187 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.441913:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972384 is committed 00002000:00010000:3.0:1789760738.442106:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20742144 -> 20746240 00010000:00010000:3.0:1789760738.442124:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55187 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.442138:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55187 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.442153:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55187 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.442162:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760738.442167:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55187 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.442176:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55187 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.442200:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55187 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.442208:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55187 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20742144->20746239] (req 20742144->20746239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.442357:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508328832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27838us (28147us total) trans 4294972384 rc 0/0 00000100:00100000:3.0:1789760738.442368:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5096 00000100:00100000:3.0:1789760738.442379:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717481831915 time 717519206435 state 0x2 ref 0 00000100:00100000:3.0:1789760738.446631:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760738.446707:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508328960 00000100:00100000:0.0:1789760738.446718:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508328960 00000100:00100000:0.0:1789760738.446779:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717481831915 time 717519206435 state 0x0 ref 1 00000100:00100000:0.0:1789760738.446795:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717519466851 time 717519206435 state 0x1 ref 1 00000100:00100000:0.0:1789760738.446827:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717519466851 time 717551537714 state 0x0 ref 1 00000100:00100000:0.0:1789760738.446842:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508328960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760738.446894:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20746240, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:0.0:1789760738.446904:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5518e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.446921:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760738.446926:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760738.446933:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5518e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760738.446949:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5518e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.533643:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972385 is committed 00002000:00010000:3.0:1789760738.533860:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20746240 -> 20750336 00010000:00010000:3.0:1789760738.533876:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5518e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.533896:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5518e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.533905:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5518e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.533912:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760738.533917:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5518e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.533926:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5518e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.533950:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5518e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.533958:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5518e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20746240->20750335] (req 20746240->20750335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.534060:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508328960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 87215us (87436us total) trans 4294972385 rc 0/0 00000100:00100000:3.0:1789760738.534070:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5097 00000100:00100000:3.0:1789760738.534081:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717519466851 time 717551537714 state 0x2 ref 0 00000100:00100000:1.0:1789760738.544735:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760738.545070:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508329088 00000100:00100000:2.0:1789760738.545082:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508329088 00000100:00100000:2.0:1789760738.545161:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717519466851 time 717551537714 state 0x0 ref 1 00000100:00100000:2.0:1789760738.545178:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717551798130 time 717551537714 state 0x1 ref 1 00000100:00100000:2.0:1789760738.545209:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717551798130 time 717649919349 state 0x0 ref 1 00000100:00100000:2.0:1789760738.545222:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508329088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760738.545278:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20750336, mtime: 0x6aad94e2, atime: 0x0, ctime: 0x6aad94e2, blocks: 0xa007 00010000:00010000:2.0:1789760738.545516:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55195 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.545535:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.545539:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.545547:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55195 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.545563:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55195 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.600984:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972386 is committed 00002000:00010000:3.0:1789760738.601237:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20750336 -> 20754432 00002000:00010000:3.0:1789760738.601242:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760738 -> 1789760739 00002000:00010000:3.0:1789760738.601245:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760738 -> 1789760739 00010000:00010000:3.0:1789760738.601260:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55195 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.601275:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55195 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.601322:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55195 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.601331:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760738.601336:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55195 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.601347:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55195 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.601371:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55195 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.601379:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55195 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20750336->20754431] (req 20750336->20754431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.601491:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508329088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56264us (56760us total) trans 4294972386 rc 0/0 00000100:00100000:3.0:1789760738.601503:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5098 00000100:00100000:3.0:1789760738.601516:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717551798130 time 717649919349 state 0x2 ref 0 00000100:00100000:2.0:1789760738.610653:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760738.610912:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508329216 00000100:00100000:3.0:1789760738.610923:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508329216 00000100:00100000:2.0:1789760738.611261:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717551798130 time 717649919349 state 0x0 ref 1 00000100:00100000:2.0:1789760738.611278:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717650179765 time 717649919349 state 0x1 ref 1 00000100:00100000:2.0:1789760738.611372:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717650179765 time 717716081172 state 0x0 ref 1 00000100:00100000:2.0:1789760738.611387:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508329216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760738.611442:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20754432, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:2.0:1789760738.611514:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5519c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.611533:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.611537:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.611545:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5519c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.611562:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5519c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.640439:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972387 is committed 00002000:00010000:3.0:1789760738.640774:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20754432 -> 20758528 00010000:00010000:3.0:1789760738.640792:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5519c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.640806:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5519c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.640815:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5519c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.640823:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760738.640829:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5519c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.640838:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5519c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.640861:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5519c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.640870:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5519c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20754432->20758527] (req 20754432->20758527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.640974:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508329216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29582us (30324us total) trans 4294972387 rc 0/0 00000100:00100000:3.0:1789760738.640984:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5099 00000100:00100000:3.0:1789760738.640995:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717650179765 time 717716081172 state 0x2 ref 0 00000100:00100000:0.0:1789760738.645838:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760738.645977:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508329344 00000100:00100000:3.0:1789760738.645988:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508329344 00000100:00100000:3.0:1789760738.646045:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717650179765 time 717716081172 state 0x0 ref 1 00000100:00100000:3.0:1789760738.646060:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717716341588 time 717716081172 state 0x1 ref 1 00000100:00100000:3.0:1789760738.646086:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717716341588 time 717750797458 state 0x0 ref 1 00000100:00100000:3.0:1789760738.646107:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508329344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760738.646158:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20758528, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:3.0:1789760738.646168:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.646186:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760738.646191:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.646198:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.646215:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.674334:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972388 is committed 00002000:00010000:3.0:1789760738.674680:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20758528 -> 20762624 00010000:00010000:3.0:1789760738.674696:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.674710:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.674719:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.674727:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760738.674732:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.674742:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.674765:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.674773:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20758528->20762623] (req 20758528->20762623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.674872:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508329344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28762us (29039us total) trans 4294972388 rc 0/0 00000100:00100000:3.0:1789760738.674883:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5100 00000100:00100000:3.0:1789760738.674894:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717716341588 time 717750797458 state 0x2 ref 0 00000100:00100000:1.0:1789760738.678753:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760738.678823:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508329472 00000100:00100000:3.0:1789760738.678834:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508329472 00000100:00100000:3.0:1789760738.678894:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717716341588 time 717750797458 state 0x0 ref 1 00000100:00100000:3.0:1789760738.678911:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717751057874 time 717750797458 state 0x1 ref 1 00000100:00100000:3.0:1789760738.678942:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717751057874 time 717783651918 state 0x0 ref 1 00000100:00100000:3.0:1789760738.678954:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d47c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508329472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760738.679005:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20762624, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:3.0:1789760738.679014:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe551aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.679031:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760738.679036:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.679044:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe551aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.679061:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe551aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760738.722206:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20762624 -> 20766720 00010000:00010000:2.0:1789760738.722226:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe551aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.722238:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe551aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.722247:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe551aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.722254:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760738.722260:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe551aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.722269:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe551aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.722312:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe551aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.722320:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe551aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20762624->20766719] (req 20762624->20766719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.722402:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d47c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508329472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43444us (43653us total) trans 4294972389 rc 0/0 00000100:00100000:2.0:1789760738.722411:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5101 00000100:00100000:2.0:1789760738.722422:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717751057874 time 717783651918 state 0x2 ref 0 00000001:00080000:0.0:1789760738.722670:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972389 is committed 00000100:00100000:2.0:1789760738.728371:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760738.728542:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508329600 00000100:00100000:3.0:1789760738.728553:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508329600 00000100:00100000:3.0:1789760738.728609:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717751057874 time 717783651918 state 0x0 ref 1 00000100:00100000:3.0:1789760738.728625:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717783912334 time 717783651918 state 0x1 ref 1 00000100:00100000:3.0:1789760738.728657:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717783912334 time 717833367416 state 0x0 ref 1 00000100:00100000:3.0:1789760738.728673:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508329600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760738.728722:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20766720, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:3.0:1789760738.728732:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.728749:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760738.728753:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.728760:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.728775:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.762972:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972390 is committed 00002000:00010000:2.0:1789760738.763078:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20766720 -> 20770816 00010000:00010000:2.0:1789760738.763106:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.763124:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.763134:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.763141:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760738.763147:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.763156:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.763178:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.763185:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20766720->20770815] (req 20766720->20770815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760738.763358:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508329600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34681us (34992us total) trans 4294972390 rc 0/0 00000100:00100000:2.0:1789760738.763369:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5102 00000100:00100000:2.0:1789760738.763380:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717783912334 time 717833367416 state 0x2 ref 0 00000100:00100000:0.0:1789760738.767703:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760738.767798:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508329728 00000100:00100000:3.0:1789760738.767809:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508329728 00000100:00100000:3.0:1789760738.767867:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717783912334 time 717833367416 state 0x0 ref 1 00000100:00100000:3.0:1789760738.767884:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717833627832 time 717833367416 state 0x1 ref 1 00000100:00100000:3.0:1789760738.767948:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717833627832 time 717872657589 state 0x0 ref 1 00000100:00100000:3.0:1789760738.767962:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508329728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760738.768083:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20770816, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:3.0:1789760738.768102:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.768125:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760738.768130:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.768138:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.768153:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760738.795260:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20770816 -> 20774912 00010000:00010000:3.0:1789760738.795282:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.795313:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.795323:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.795331:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760738.795337:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.795347:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.795373:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.795382:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20770816->20774911] (req 20770816->20774911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.795492:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508329728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27526us (27793us total) trans 4294972391 rc 0/0 00000100:00100000:3.0:1789760738.795503:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5103 00000100:00100000:3.0:1789760738.795514:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717833627832 time 717872657589 state 0x2 ref 0 00000001:00080000:1.0:1789760738.795646:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972391 is committed 00000100:00100000:1.0:1789760738.802732:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760738.802879:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508329856 00000100:00100000:2.0:1789760738.802891:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508329856 00000100:00100000:2.0:1789760738.802953:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717833627832 time 717872657589 state 0x0 ref 1 00000100:00100000:2.0:1789760738.802970:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717872918005 time 717872657589 state 0x1 ref 1 00000100:00100000:2.0:1789760738.803038:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717872918005 time 717907747264 state 0x0 ref 1 00000100:00100000:2.0:1789760738.803052:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508329856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760738.803113:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20774912, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:2.0:1789760738.803123:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe551bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.803142:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.803147:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.803156:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe551bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.803172:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe551bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.864018:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972392 is committed 00002000:00010000:3.0:1789760738.864130:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20774912 -> 20779008 00010000:00010000:3.0:1789760738.864147:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe551bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.864159:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe551bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.864168:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe551bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.864176:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760738.864181:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe551bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.864191:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe551bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.864218:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe551bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.864227:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe551bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20774912->20779007] (req 20774912->20779007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.864367:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508329856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61311us (61639us total) trans 4294972392 rc 0/0 00000100:00100000:3.0:1789760738.864378:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5104 00000100:00100000:3.0:1789760738.864390:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717872918005 time 717907747264 state 0x2 ref 0 00000100:00100000:0.0:1789760738.868485:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760738.868663:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508329984 00000100:00100000:2.0:1789760738.868674:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508329984 00000100:00100000:2.0:1789760738.868734:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717872918005 time 717907747264 state 0x0 ref 1 00000100:00100000:2.0:1789760738.868751:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717908007680 time 717907747264 state 0x1 ref 1 00000100:00100000:2.0:1789760738.868785:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717908007680 time 717973494525 state 0x0 ref 1 00000100:00100000:2.0:1789760738.868802:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508329984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760738.868854:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20779008, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:2.0:1789760738.868864:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe551c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.868881:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760738.868886:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760738.868894:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe551c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760738.868910:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe551c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760738.922787:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972393 is committed 00002000:00010000:3.0:1789760738.922884:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20779008 -> 20783104 00010000:00010000:3.0:1789760738.922901:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe551c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.922914:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe551c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.922923:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe551c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.922931:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760738.922937:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe551c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.922953:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe551c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.922980:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe551c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.922988:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe551c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20779008->20783103] (req 20779008->20783103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.923208:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508329984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54402us (54727us total) trans 4294972393 rc 0/0 00000100:00100000:3.0:1789760738.923220:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5105 00000100:00100000:3.0:1789760738.923231:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717908007680 time 717973494525 state 0x2 ref 0 00000100:00100000:0.0:1789760738.926674:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760738.926754:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508330112 00000100:00100000:3.0:1789760738.926764:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508330112 00000100:00100000:3.0:1789760738.926822:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717908007680 time 717973494525 state 0x0 ref 1 00000100:00100000:3.0:1789760738.926839:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717973754941 time 717973494525 state 0x1 ref 1 00000100:00100000:3.0:1789760738.926872:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717973754941 time 718031582074 state 0x0 ref 1 00000100:00100000:3.0:1789760738.926885:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508330112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760738.926936:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20783104, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:3.0:1789760738.926946:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.926965:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760738.926970:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.926978:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.926994:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760738.965108:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972394 is committed 00002000:00010000:3.0:1789760738.965355:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20783104 -> 20787200 00010000:00010000:3.0:1789760738.965372:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.965384:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.965392:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.965399:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760738.965404:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.965412:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.965436:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.965444:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20783104->20787199] (req 20783104->20787199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760738.965607:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508330112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38718us (38938us total) trans 4294972394 rc 0/0 00000100:00100000:3.0:1789760738.965617:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5106 00000100:00100000:3.0:1789760738.965628:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717973754941 time 718031582074 state 0x2 ref 0 00000100:00100000:0.0:1789760738.969737:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760738.969832:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508330240 00000100:00100000:3.0:1789760738.969843:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508330240 00000100:00100000:3.0:1789760738.969903:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 717973754941 time 718031582074 state 0x0 ref 1 00000100:00100000:3.0:1789760738.969921:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718031842490 time 718031582074 state 0x1 ref 1 00000100:00100000:3.0:1789760738.969991:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718031842490 time 718074700576 state 0x0 ref 1 00000100:00100000:3.0:1789760738.970017:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508330240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760738.970072:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20787200, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:3.0:1789760738.970082:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.970109:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760738.970114:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760738.970121:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760738.970143:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760739.035811:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20787200 -> 20791296 00010000:00010000:3.0:1789760739.035834:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.035846:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.035855:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.035863:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760739.035868:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.035877:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.035900:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.035907:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20787200->20791295] (req 20787200->20791295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.036000:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508330240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 65992us (66267us total) trans 4294972395 rc 0/0 00000001:00080000:1.0:1789760739.036183:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972395 is committed 00000100:00100000:3.0:1789760739.036234:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5107 00000100:00100000:3.0:1789760739.036246:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718031842490 time 718074700576 state 0x2 ref 0 00000100:00100000:2.0:1789760739.039472:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760739.039585:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508330368 00000100:00100000:3.0:1789760739.039595:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508330368 00000100:00100000:3.0:1789760739.039668:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718031842490 time 718074700576 state 0x0 ref 1 00000100:00100000:3.0:1789760739.039684:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718074960992 time 718074700576 state 0x1 ref 1 00000100:00100000:3.0:1789760739.039714:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718074960992 time 718144423996 state 0x0 ref 1 00000100:00100000:3.0:1789760739.039728:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508330368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760739.039781:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20791296, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:3.0:1789760739.039791:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.039810:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760739.039814:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.039822:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.039839:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760739.040144:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760739.061234:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20791296 -> 20795392 00010000:00010000:2.0:1789760739.061256:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.061269:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.061278:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.061305:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760739.061310:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.061321:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.061346:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.061354:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe551db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20791296->20795391] (req 20791296->20795391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.061463:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508330368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 21732us (21995us total) trans 4294972396 rc 0/0 00000100:00100000:2.0:1789760739.061474:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5108 00000100:00100000:2.0:1789760739.061485:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718074960992 time 718144423996 state 0x2 ref 0 00000001:00080000:3.0:1789760739.061608:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972396 is committed 00000100:00100000:2.0:1789760739.064850:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760739.064934:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508330496 00000100:00100000:3.0:1789760739.064945:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508330496 00000100:00100000:3.0:1789760739.065000:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718074960992 time 718144423996 state 0x0 ref 1 00000100:00100000:3.0:1789760739.065018:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718144684412 time 718144423996 state 0x1 ref 1 00000100:00100000:3.0:1789760739.065056:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718144684412 time 718169766393 state 0x0 ref 1 00000100:00100000:3.0:1789760739.065070:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508330496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760739.065128:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20795392, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:3.0:1789760739.065138:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.065154:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760739.065159:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.065166:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.065181:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.073564:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760739.088682:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972397 is committed 00002000:00010000:3.0:1789760739.088787:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20795392 -> 20799488 00010000:00010000:3.0:1789760739.088804:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.088817:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.088826:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.088833:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760739.088838:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.088848:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.088871:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.088879:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe551e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20795392->20799487] (req 20795392->20799487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.088979:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508330496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23905us (24132us total) trans 4294972397 rc 0/0 00000100:00100000:3.0:1789760739.088990:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5109 00000100:00100000:3.0:1789760739.089000:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718144684412 time 718169766393 state 0x2 ref 0 00000100:00100000:1.0:1789760739.092496:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760739.092579:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508330624 00000100:00100000:3.0:1789760739.092589:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508330624 00000100:00100000:3.0:1789760739.092649:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718144684412 time 718169766393 state 0x0 ref 1 00000100:00100000:3.0:1789760739.092665:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718170026809 time 718169766393 state 0x1 ref 1 00000100:00100000:3.0:1789760739.092727:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718170026809 time 718197436365 state 0x0 ref 1 00000100:00100000:3.0:1789760739.092740:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508330624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760739.092790:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20799488, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:3.0:1789760739.092799:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.092817:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760739.092821:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.092827:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.092842:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760739.122259:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20799488 -> 20803584 00010000:00010000:2.0:1789760739.122281:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.122324:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.122333:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.122345:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760739.122350:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.122360:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.122384:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.122392:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe551e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20799488->20803583] (req 20799488->20803583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.122500:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508330624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29757us (30065us total) trans 4294972398 rc 0/0 00000100:00100000:2.0:1789760739.122511:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5110 00000100:00100000:2.0:1789760739.122521:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718170026809 time 718197436365 state 0x2 ref 0 00000001:00080000:3.0:1789760739.125639:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972398 is committed 00000100:00100000:2.0:1789760739.126277:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760739.126423:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508330752 00000100:00100000:0.0:1789760739.126433:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508330752 00000100:00100000:0.0:1789760739.126497:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718170026809 time 718197436365 state 0x0 ref 1 00000100:00100000:0.0:1789760739.126513:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718197696781 time 718197436365 state 0x1 ref 1 00000100:00100000:0.0:1789760739.126542:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718197696781 time 718231252059 state 0x0 ref 1 00000100:00100000:0.0:1789760739.126554:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508330752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760739.126603:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20803584, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:0.0:1789760739.126612:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe551f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.126628:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760739.126633:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.126641:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe551f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.126657:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe551f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.148403:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972399 is committed 00002000:00010000:3.0:1789760739.148592:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20803584 -> 20807680 00010000:00010000:3.0:1789760739.148609:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe551f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.148621:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe551f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.148636:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe551f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.148644:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760739.148649:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe551f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.148659:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe551f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.148681:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe551f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.148690:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe551f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20803584->20807679] (req 20803584->20807679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.148782:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508330752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22223us (22505us total) trans 4294972399 rc 0/0 00000100:00100000:3.0:1789760739.148792:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5111 00000100:00100000:3.0:1789760739.148803:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718197696781 time 718231252059 state 0x2 ref 0 00000100:00100000:1.0:1789760739.151863:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760739.152233:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508330880 00000100:00100000:3.0:1789760739.152244:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508330880 00000100:00100000:2.0:1789760739.152406:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718197696781 time 718231252059 state 0x0 ref 1 00000100:00100000:2.0:1789760739.152424:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718231512475 time 718231252059 state 0x1 ref 1 00000100:00100000:2.0:1789760739.152551:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718231512475 time 718257259985 state 0x0 ref 1 00000100:00100000:2.0:1789760739.152568:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508330880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760739.152622:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20807680, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:2.0:1789760739.152631:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe551f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.152649:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.152654:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.152662:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe551f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.152676:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe551f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.179042:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972400 is committed 00002000:00010000:2.0:1789760739.179225:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20807680 -> 20811776 00010000:00010000:2.0:1789760739.179243:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe551f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.179256:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe551f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.179265:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe551f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.179272:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760739.179277:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe551f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.179305:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe551f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.179330:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe551f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.179339:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe551f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20807680->20811775] (req 20807680->20811775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.179477:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508330880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26905us (27619us total) trans 4294972400 rc 0/0 00000100:00100000:2.0:1789760739.179488:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5112 00000100:00100000:2.0:1789760739.179499:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718231512475 time 718257259985 state 0x2 ref 0 00000100:00100000:1.0:1789760739.183219:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760739.183355:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508331008 00000100:00100000:2.0:1789760739.183365:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508331008 00000100:00100000:2.0:1789760739.183417:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718231512475 time 718257259985 state 0x0 ref 1 00000100:00100000:2.0:1789760739.183432:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718257520401 time 718257259985 state 0x1 ref 1 00000100:00100000:2.0:1789760739.183534:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718257520401 time 718288243792 state 0x0 ref 1 00000100:00100000:2.0:1789760739.183547:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508331008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760739.183594:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20811776, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:2.0:1789760739.183604:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe551fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.183620:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.183625:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.183631:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe551fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.183645:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe551fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.200844:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049beafaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534068736:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760739.200958:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760739.201049:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534068736 00000100:00100000:2.0:1789760739.201057:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534068736 00000100:00100000:2.0:1789760739.201113:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 429 00000100:00100000:2.0:1789760739.201122:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534068736:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760739.201162:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789760739.201210:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534068736:12345-0@lo:41:osp-pre-1-0.0 Request processed in 89us (251us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760739.201217:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 429 00000100:00100000:2.0:1789760739.201265:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049beafaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534068736:0@lo:41:osp-pre-1-0.0 00002000:00010000:2.0:1789760739.211695:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20811776 -> 20815872 00010000:00010000:2.0:1789760739.211714:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe551fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.211726:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe551fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.211740:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe551fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.211748:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760739.211753:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe551fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.211762:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe551fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.211782:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe551fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.211790:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe551fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20811776->20815871] (req 20811776->20815871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.211885:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508331008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28334us (28669us total) trans 4294972401 rc 0/0 00000100:00100000:2.0:1789760739.211896:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5113 00000100:00100000:2.0:1789760739.211908:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718257520401 time 718288243792 state 0x2 ref 0 00000001:00080000:3.0:1789760739.214873:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972401 is committed 00000100:00100000:0.0:1789760739.214986:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760739.215122:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508331136 00000100:00100000:3.0:1789760739.215132:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508331136 00000100:00100000:3.0:1789760739.215186:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718257520401 time 718288243792 state 0x0 ref 1 00000100:00100000:3.0:1789760739.215203:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718288504208 time 718288243792 state 0x1 ref 1 00000100:00100000:3.0:1789760739.215233:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718288504208 time 718319943409 state 0x0 ref 1 00000100:00100000:3.0:1789760739.215246:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508331136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760739.215388:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20815872, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:3.0:1789760739.215398:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55205 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.215414:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760739.215419:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.215426:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55205 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.215499:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55205 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.238705:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972402 is committed 00002000:00010000:1.0:1789760739.239550:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20815872 -> 20819968 00010000:00010000:1.0:1789760739.239567:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55205 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.239580:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55205 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.239589:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55205 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.239598:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760739.239602:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55205 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.239612:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55205 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.239635:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55205 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.239643:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55205 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20815872->20819967] (req 20815872->20819967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.239744:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508331136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24494us (24757us total) trans 4294972402 rc 0/0 00000100:00100000:1.0:1789760739.239755:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5114 00000100:00100000:1.0:1789760739.239766:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718288504208 time 718319943409 state 0x2 ref 0 00000100:00100000:1.0:1789760739.243676:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760739.243775:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508331264 00000100:00100000:2.0:1789760739.243786:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508331264 00000100:00100000:2.0:1789760739.243841:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718288504208 time 718319943409 state 0x0 ref 1 00000100:00100000:2.0:1789760739.243861:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718320203825 time 718319943409 state 0x1 ref 1 00000100:00100000:2.0:1789760739.243917:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718320203825 time 718348626762 state 0x0 ref 1 00000100:00100000:2.0:1789760739.243930:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508331264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760739.243979:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20819968, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:2.0:1789760739.243989:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5520c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.244018:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.244023:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.244030:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5520c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.244046:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5520c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.273260:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972403 is committed 00002000:00010000:2.0:1789760739.273348:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20819968 -> 20824064 00010000:00010000:2.0:1789760739.273364:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5520c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.273377:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5520c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.273386:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5520c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.273394:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760739.273400:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5520c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.273409:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5520c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.273433:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5520c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.273455:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5520c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20819968->20824063] (req 20819968->20824063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.273542:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508331264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29608us (29869us total) trans 4294972403 rc 0/0 00000100:00100000:2.0:1789760739.273554:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5115 00000100:00100000:2.0:1789760739.273565:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718320203825 time 718348626762 state 0x2 ref 0 00000100:00100000:3.0:1789760739.276809:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760739.276887:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508331392 00000100:00100000:2.0:1789760739.276898:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508331392 00000100:00100000:2.0:1789760739.276954:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718320203825 time 718348626762 state 0x0 ref 1 00000100:00100000:2.0:1789760739.276971:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718348887178 time 718348626762 state 0x1 ref 1 00000100:00100000:2.0:1789760739.277013:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718348887178 time 718381714001 state 0x0 ref 1 00000100:00100000:2.0:1789760739.277029:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508331392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760739.277078:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20824064, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:2.0:1789760739.277087:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55213 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.277116:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.277121:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.277129:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55213 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.277144:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55213 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760739.307209:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20824064 -> 20828160 00010000:00010000:2.0:1789760739.307229:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55213 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.307242:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55213 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.307251:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55213 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.307259:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760739.307264:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55213 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.307274:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55213 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.307317:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55213 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.307326:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55213 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20824064->20828159] (req 20824064->20828159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.307422:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508331392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30388us (30617us total) trans 4294972404 rc 0/0 00000100:00100000:2.0:1789760739.307432:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5116 00000100:00100000:2.0:1789760739.307453:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718348887178 time 718381714001 state 0x2 ref 0 00000001:00080000:1.0:1789760739.307576:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972404 is committed 00000100:00100000:3.0:1789760739.311000:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760739.311117:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508331520 00000100:00100000:1.0:1789760739.311127:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508331520 00000100:00100000:1.0:1789760739.311184:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718348887178 time 718381714001 state 0x0 ref 1 00000100:00100000:1.0:1789760739.311201:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718381974417 time 718381714001 state 0x1 ref 1 00000100:00100000:1.0:1789760739.311242:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718381974417 time 718415951550 state 0x0 ref 1 00000100:00100000:1.0:1789760739.311255:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508331520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760739.311340:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20828160, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:1.0:1789760739.311349:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5521a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.311366:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760739.311371:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.311378:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5521a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.311393:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5521a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760739.334601:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20828160 -> 20832256 00010000:00010000:3.0:1789760739.334623:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5521a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.334636:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5521a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.334647:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5521a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.334655:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760739.334660:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5521a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.334671:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5521a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.334695:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5521a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.334704:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5521a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20828160->20832255] (req 20828160->20832255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.334804:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508331520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23544us (23809us total) trans 4294972405 rc 0/0 00000100:00100000:3.0:1789760739.334816:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5117 00000100:00100000:3.0:1789760739.334827:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718381974417 time 718415951550 state 0x2 ref 0 00000001:00080000:0.0:1789760739.337766:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972405 is committed 00000100:00100000:0.0:1789760739.338238:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760739.338330:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508331648 00000100:00100000:2.0:1789760739.338341:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508331648 00000100:00100000:2.0:1789760739.338400:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718381974417 time 718415951550 state 0x0 ref 1 00000100:00100000:2.0:1789760739.338418:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718416211966 time 718415951550 state 0x1 ref 1 00000100:00100000:2.0:1789760739.338456:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718416211966 time 718443166132 state 0x0 ref 1 00000100:00100000:2.0:1789760739.338469:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508331648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760739.338524:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20832256, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:2.0:1789760739.338534:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55221 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.338552:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.338557:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.338564:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55221 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.338579:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55221 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760739.372346:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20832256 -> 20836352 00010000:00010000:0.0:1789760739.372367:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55221 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.372380:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55221 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.372389:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55221 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.372397:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760739.372402:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55221 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.372412:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55221 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.372435:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55221 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.372461:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55221 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20832256->20836351] (req 20832256->20836351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.372553:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508331648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34080us (34314us total) trans 4294972406 rc 0/0 00000100:00100000:0.0:1789760739.372565:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5118 00000100:00100000:0.0:1789760739.372576:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718416211966 time 718443166132 state 0x2 ref 0 00000001:00080000:0.0:1789760739.372615:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972406 is committed 00000100:00100000:2.0:1789760739.376505:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760739.376606:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508331776 00000100:00100000:3.0:1789760739.376616:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508331776 00000100:00100000:3.0:1789760739.376673:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718416211966 time 718443166132 state 0x0 ref 1 00000100:00100000:3.0:1789760739.376689:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718443426548 time 718443166132 state 0x1 ref 1 00000100:00100000:3.0:1789760739.376751:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718443426548 time 718481460386 state 0x0 ref 1 00000100:00100000:3.0:1789760739.376764:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508331776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760739.376812:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20836352, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:3.0:1789760739.376822:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55228 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.376841:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760739.376845:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.376853:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55228 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.376870:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55228 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.401414:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972407 is committed 00002000:00010000:2.0:1789760739.401560:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20836352 -> 20840448 00010000:00010000:2.0:1789760739.401576:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55228 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.401589:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55228 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.401598:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55228 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.401606:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760739.401611:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55228 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.401620:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55228 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.401648:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55228 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.401669:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55228 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20836352->20840447] (req 20836352->20840447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.401769:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508331776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25002us (25324us total) trans 4294972407 rc 0/0 00000100:00100000:2.0:1789760739.401780:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5119 00000100:00100000:2.0:1789760739.401791:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718443426548 time 718481460386 state 0x2 ref 0 00000100:00100000:1.0:1789760739.405458:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760739.405608:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508331904 00000100:00100000:3.0:1789760739.405620:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508331904 00000100:00100000:3.0:1789760739.405689:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718443426548 time 718481460386 state 0x0 ref 1 00000100:00100000:3.0:1789760739.405707:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718481720802 time 718481460386 state 0x1 ref 1 00000100:00100000:3.0:1789760739.405738:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718481720802 time 718510447848 state 0x0 ref 1 00000100:00100000:3.0:1789760739.405750:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508331904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760739.405803:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20840448, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:3.0:1789760739.405826:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5522f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.405847:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760739.405852:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.405860:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5522f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.405876:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5522f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760739.431311:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20840448 -> 20844544 00010000:00010000:1.0:1789760739.431333:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5522f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.431347:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5522f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.431356:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5522f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.431371:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760739.431377:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5522f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.431387:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5522f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.431415:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5522f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.431423:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5522f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20840448->20844543] (req 20840448->20844543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.431536:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508331904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25781us (26090us total) trans 4294972408 rc 0/0 00000100:00100000:1.0:1789760739.431547:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5120 00000100:00100000:1.0:1789760739.431558:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718481720802 time 718510447848 state 0x2 ref 0 00000001:00080000:0.0:1789760739.431698:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972408 is committed 00000100:00100000:0.0:1789760739.434799:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760739.435028:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508332032 00000100:00100000:1.0:1789760739.435039:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508332032 00000100:00100000:1.0:1789760739.435104:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718481720802 time 718510447848 state 0x0 ref 1 00000100:00100000:1.0:1789760739.435121:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718510708264 time 718510447848 state 0x1 ref 1 00000100:00100000:1.0:1789760739.435153:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718510708264 time 718539863008 state 0x0 ref 1 00000100:00100000:1.0:1789760739.435168:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508332032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760739.435219:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20844544, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:1.0:1789760739.435229:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55236 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.435247:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760739.435251:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.435258:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55236 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.435273:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55236 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760739.464946:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972409 is committed 00002000:00010000:1.0:1789760739.465252:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20844544 -> 20848640 00010000:00010000:1.0:1789760739.465269:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55236 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.465282:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55236 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.465324:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55236 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.465331:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760739.465336:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55236 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.465345:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55236 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.465367:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55236 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.465375:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55236 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20844544->20848639] (req 20844544->20848639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.465486:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508332032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30313us (30689us total) trans 4294972409 rc 0/0 00000100:00100000:1.0:1789760739.465497:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5121 00000100:00100000:1.0:1789760739.465508:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718510708264 time 718539863008 state 0x2 ref 0 00000100:00100000:3.0:1789760739.468640:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760739.468947:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508332160 00000100:00100000:2.0:1789760739.468959:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508332160 00000100:00100000:2.0:1789760739.469029:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718510708264 time 718539863008 state 0x0 ref 1 00000100:00100000:2.0:1789760739.469046:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718540123424 time 718539863008 state 0x1 ref 1 00000100:00100000:2.0:1789760739.469077:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718540123424 time 718573787941 state 0x0 ref 1 00000100:00100000:2.0:1789760739.469099:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508332160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760739.469149:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20848640, mtime: 0x6aad94e3, atime: 0x0, ctime: 0x6aad94e3, blocks: 0xa007 00010000:00010000:2.0:1789760739.469159:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5523d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.469176:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.469181:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.469188:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5523d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.469204:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5523d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.490883:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972410 is committed 00002000:00010000:2.0:1789760739.491133:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20848640 -> 20852736 00002000:00010000:2.0:1789760739.491138:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760739 -> 1789760740 00002000:00010000:2.0:1789760739.491140:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760739 -> 1789760740 00010000:00010000:2.0:1789760739.491155:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5523d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.491166:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5523d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.491175:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5523d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.491182:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760739.491187:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5523d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.491196:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5523d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.491219:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5523d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.491228:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5523d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20848640->20852735] (req 20848640->20852735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.491366:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508332160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22271us (22728us total) trans 4294972410 rc 0/0 00000100:00100000:2.0:1789760739.491376:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5122 00000100:00100000:2.0:1789760739.491387:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718540123424 time 718573787941 state 0x2 ref 0 00000100:00100000:3.0:1789760739.507532:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760739.507684:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508332288 00000100:00100000:1.0:1789760739.507695:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508332288 00000100:00100000:1.0:1789760739.507756:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718540123424 time 718573787941 state 0x0 ref 1 00000100:00100000:1.0:1789760739.507773:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718574048357 time 718573787941 state 0x1 ref 1 00000100:00100000:1.0:1789760739.507804:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718574048357 time 718612514552 state 0x0 ref 1 00000100:00100000:1.0:1789760739.507818:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508332288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760739.507934:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20852736, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:1.0:1789760739.507945:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55244 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.507964:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760739.507969:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.507976:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55244 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.507991:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55244 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.541372:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972411 is committed 00002000:00010000:3.0:1789760739.541559:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20852736 -> 20856832 00010000:00010000:3.0:1789760739.541575:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55244 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.541587:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55244 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.541596:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55244 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.541604:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760739.541609:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55244 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.541624:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55244 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.541646:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55244 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.541655:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55244 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20852736->20856831] (req 20852736->20856831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.541756:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508332288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33934us (34228us total) trans 4294972411 rc 0/0 00000100:00100000:3.0:1789760739.541767:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5123 00000100:00100000:3.0:1789760739.541778:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718574048357 time 718612514552 state 0x2 ref 0 00000100:00100000:3.0:1789760739.546850:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760739.547018:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508332416 00000100:00100000:1.0:1789760739.547030:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508332416 00000100:00100000:1.0:1789760739.547103:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718574048357 time 718612514552 state 0x0 ref 1 00000100:00100000:1.0:1789760739.547121:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718612774968 time 718612514552 state 0x1 ref 1 00000100:00100000:1.0:1789760739.547153:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718612774968 time 718651863706 state 0x0 ref 1 00000100:00100000:1.0:1789760739.547165:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508332416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760739.547219:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20856832, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:1.0:1789760739.547229:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5524b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.547245:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760739.547249:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.547256:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5524b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.547271:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5524b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.595511:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972412 is committed 00002000:00010000:1.0:1789760739.595520:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20856832 -> 20860928 00010000:00010000:1.0:1789760739.595540:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5524b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.595555:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5524b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.595565:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5524b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.595573:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760739.595578:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5524b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.595587:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5524b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.595621:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5524b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.595633:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5524b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20856832->20860927] (req 20856832->20860927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.595748:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508332416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48578us (48905us total) trans 4294972412 rc 0/0 00000100:00100000:1.0:1789760739.595759:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5124 00000100:00100000:1.0:1789760739.595770:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718612774968 time 718651863706 state 0x2 ref 0 00000100:00100000:2.0:1789760739.600798:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760739.601956:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508332544 00000100:00100000:0.0:1789760739.601969:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508332544 00000100:00100000:1.0:1789760739.602156:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718612774968 time 718651863706 state 0x0 ref 1 00000100:00100000:1.0:1789760739.602174:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718652124122 time 718651863706 state 0x1 ref 1 00000100:00100000:1.0:1789760739.602260:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718652124122 time 718706968972 state 0x0 ref 1 00000100:00100000:1.0:1789760739.602277:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508332544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760739.602353:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20860928, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:1.0:1789760739.602363:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55252 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.602392:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760739.602397:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.602403:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55252 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.602420:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55252 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.646851:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972413 is committed 00002000:00010000:3.0:1789760739.646919:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20860928 -> 20865024 00010000:00010000:3.0:1789760739.646936:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55252 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.646949:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55252 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.646957:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55252 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.646964:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760739.646970:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55252 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.646980:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55252 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.647014:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55252 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.647024:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55252 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20860928->20865023] (req 20860928->20865023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.647132:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508332544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44853us (46339us total) trans 4294972413 rc 0/0 00000100:00100000:3.0:1789760739.647143:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5125 00000100:00100000:3.0:1789760739.647153:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718652124122 time 718706968972 state 0x2 ref 0 00000100:00100000:3.0:1789760739.650861:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760739.650954:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508332672 00000100:00100000:2.0:1789760739.650966:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508332672 00000100:00100000:2.0:1789760739.651026:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718652124122 time 718706968972 state 0x0 ref 1 00000100:00100000:2.0:1789760739.651044:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718707229388 time 718706968972 state 0x1 ref 1 00000100:00100000:2.0:1789760739.651111:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718707229388 time 718755820500 state 0x0 ref 1 00000100:00100000:2.0:1789760739.651124:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508332672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760739.651175:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20865024, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:2.0:1789760739.651185:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55259 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.651203:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.651209:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.651217:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55259 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.651233:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55259 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.675416:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972414 is committed 00002000:00010000:2.0:1789760739.675630:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20865024 -> 20869120 00010000:00010000:2.0:1789760739.675647:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55259 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.675662:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55259 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.675672:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55259 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.675680:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760739.675685:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55259 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.675695:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55259 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.675718:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55259 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.675727:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55259 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20865024->20869119] (req 20865024->20869119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.675944:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508332672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24817us (25087us total) trans 4294972414 rc 0/0 00000100:00100000:2.0:1789760739.675956:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5126 00000100:00100000:2.0:1789760739.675966:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718707229388 time 718755820500 state 0x2 ref 0 00000100:00100000:0.0:1789760739.683218:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760739.683329:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508332800 00000100:00100000:1.0:1789760739.683341:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508332800 00000100:00100000:1.0:1789760739.683402:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718707229388 time 718755820500 state 0x0 ref 1 00000100:00100000:1.0:1789760739.683418:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718756080916 time 718755820500 state 0x1 ref 1 00000100:00100000:1.0:1789760739.683459:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718756080916 time 718788169311 state 0x0 ref 1 00000100:00100000:1.0:1789760739.683472:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508332800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760739.683524:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20869120, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:1.0:1789760739.683535:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55260 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.683551:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760739.683556:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.683563:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55260 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.683577:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55260 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760739.722965:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20869120 -> 20873216 00010000:00010000:1.0:1789760739.722985:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55260 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.723617:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972415 is committed 00010000:00010000:1.0:1789760739.723688:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55260 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.723706:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55260 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.723714:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760739.723720:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55260 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.723730:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55260 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.723765:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55260 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.723774:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55260 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20869120->20873215] (req 20869120->20873215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.724255:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508332800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40778us (41040us total) trans 4294972415 rc 0/0 00000100:00100000:1.0:1789760739.724267:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5127 00000100:00100000:1.0:1789760739.724278:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718756080916 time 718788169311 state 0x2 ref 0 00000100:00100000:3.0:1789760739.727423:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760739.732052:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508332928 00000100:00100000:1.0:1789760739.732068:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508332928 00000100:00100000:1.0:1789760739.732147:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718756080916 time 718788169311 state 0x0 ref 1 00000100:00100000:1.0:1789760739.732164:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718788429727 time 718788169311 state 0x1 ref 1 00000100:00100000:1.0:1789760739.732204:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718788429727 time 718836913853 state 0x0 ref 1 00000100:00100000:1.0:1789760739.732219:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508332928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760739.732273:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20873216, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:1.0:1789760739.732283:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55267 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.732314:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760739.732318:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.732325:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55267 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.732340:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55267 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.766732:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972416 is committed 00002000:00010000:3.0:1789760739.766855:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20873216 -> 20877312 00010000:00010000:3.0:1789760739.766872:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55267 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.766891:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55267 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.766901:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55267 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.766909:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760739.766914:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55267 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.766923:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55267 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.766948:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55267 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.766956:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55267 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20873216->20877311] (req 20873216->20877311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.767095:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508332928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34866us (39667us total) trans 4294972416 rc 0/0 00000100:00100000:3.0:1789760739.767113:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5128 00000100:00100000:3.0:1789760739.767125:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718788429727 time 718836913853 state 0x2 ref 0 00000100:00100000:1.0:1789760739.770713:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760739.771923:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508333056 00000100:00100000:2.0:1789760739.771936:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508333056 00000100:00100000:2.0:1789760739.772008:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718788429727 time 718836913853 state 0x0 ref 1 00000100:00100000:2.0:1789760739.772026:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718837174269 time 718836913853 state 0x1 ref 1 00000100:00100000:2.0:1789760739.772053:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718837174269 time 718876764157 state 0x0 ref 1 00000100:00100000:2.0:1789760739.772067:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508333056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760739.772130:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20877312, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:2.0:1789760739.772140:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5526e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.772160:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.772165:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.772172:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5526e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.772187:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5526e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760739.798530:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20877312 -> 20881408 00010000:00010000:3.0:1789760739.798552:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5526e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.798566:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5526e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.798575:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5526e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.798582:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760739.798588:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5526e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.798598:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5526e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.798620:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5526e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.798627:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5526e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20877312->20881407] (req 20877312->20881407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760739.798717:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508333056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26648us (28009us total) trans 4294972417 rc 0/0 00000100:00100000:3.0:1789760739.798728:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5129 00000100:00100000:3.0:1789760739.798738:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718837174269 time 718876764157 state 0x2 ref 0 00000001:00080000:0.0:1789760739.799082:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972417 is committed 00000100:00100000:1.0:1789760739.802450:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760739.802593:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508333184 00000100:00100000:0.0:1789760739.802603:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508333184 00000100:00100000:0.0:1789760739.802769:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718837174269 time 718876764157 state 0x0 ref 1 00000100:00100000:0.0:1789760739.802786:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718877024573 time 718876764157 state 0x1 ref 1 00000100:00100000:0.0:1789760739.802881:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718877024573 time 718907590850 state 0x0 ref 1 00000100:00100000:0.0:1789760739.802896:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508333184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760739.802946:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20881408, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:0.0:1789760739.802956:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55275 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.802972:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760739.802977:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.802984:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55275 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.802999:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55275 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760739.823689:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972418 is committed 00002000:00010000:0.0:1789760739.823769:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20881408 -> 20885504 00010000:00010000:0.0:1789760739.823785:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55275 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.823798:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55275 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.823806:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55275 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.823813:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760739.823818:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55275 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.823826:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55275 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.823848:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55275 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.823855:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55275 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20881408->20885503] (req 20881408->20885503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.823984:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508333184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 21085us (21547us total) trans 4294972418 rc 0/0 00000100:00100000:0.0:1789760739.823995:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5130 00000100:00100000:0.0:1789760739.824080:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718877024573 time 718907590850 state 0x2 ref 0 00000100:00100000:2.0:1789760739.828196:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760739.828540:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508333312 00000100:00100000:3.0:1789760739.828551:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508333312 00000100:00100000:2.0:1789760739.828667:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718877024573 time 718907590850 state 0x0 ref 1 00000100:00100000:2.0:1789760739.828685:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718907851266 time 718907590850 state 0x1 ref 1 00000100:00100000:2.0:1789760739.828746:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718907851266 time 718933456471 state 0x0 ref 1 00000100:00100000:2.0:1789760739.828761:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508333312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760739.828812:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20885504, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:2.0:1789760739.828822:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5527c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.828839:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760739.828855:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760739.828862:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5527c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.828878:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5527c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.853098:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972419 is committed 00002000:00010000:0.0:1789760739.853194:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20885504 -> 20889600 00010000:00010000:0.0:1789760739.853211:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5527c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.853225:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5527c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.853235:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5527c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.853243:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760739.853248:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5527c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.853261:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5527c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.853569:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5527c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.853579:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5527c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20885504->20889599] (req 20885504->20889599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760739.853731:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508333312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24968us (25538us total) trans 4294972419 rc 0/0 00000100:00100000:1.0:1789760739.853964:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5131 00000100:00100000:1.0:1789760739.853977:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718907851266 time 718933456471 state 0x2 ref 0 00000100:00100000:2.0:1789760739.861612:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760739.861698:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508333440 00000100:00100000:3.0:1789760739.861710:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508333440 00000100:00100000:3.0:1789760739.861771:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718907851266 time 718933456471 state 0x0 ref 1 00000100:00100000:3.0:1789760739.861789:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718933716887 time 718933456471 state 0x1 ref 1 00000100:00100000:3.0:1789760739.861820:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718933716887 time 718966530081 state 0x0 ref 1 00000100:00100000:3.0:1789760739.861834:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508333440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760739.861884:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20889600, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:3.0:1789760739.861891:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55283 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.861907:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760739.861910:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760739.861915:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55283 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760739.861928:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55283 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760739.898104:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972420 is committed 00002000:00010000:0.0:1789760739.898204:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20889600 -> 20893696 00010000:00010000:0.0:1789760739.898219:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55283 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.898232:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55283 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.898241:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55283 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.898248:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760739.898253:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55283 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.898261:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55283 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.898307:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55283 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.898315:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55283 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20889600->20893695] (req 20889600->20893695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.898408:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508333440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36571us (36800us total) trans 4294972420 rc 0/0 00000100:00100000:0.0:1789760739.898417:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5132 00000100:00100000:0.0:1789760739.898427:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718933716887 time 718966530081 state 0x2 ref 0 00000100:00100000:1.0:1789760739.903618:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760739.906047:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508333568 00000100:00100000:0.0:1789760739.906063:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508333568 00000100:00100000:0.0:1789760739.906140:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718933716887 time 718966530081 state 0x0 ref 1 00000100:00100000:0.0:1789760739.906156:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718966790497 time 718966530081 state 0x1 ref 1 00000100:00100000:0.0:1789760739.906191:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718966790497 time 719010901184 state 0x0 ref 1 00000100:00100000:0.0:1789760739.906204:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508333568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760739.906252:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20893696, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:0.0:1789760739.906261:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5528a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.906279:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760739.906300:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.906309:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5528a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.906323:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5528a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760739.932448:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972421 is committed 00002000:00010000:0.0:1789760739.932669:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20893696 -> 20897792 00010000:00010000:0.0:1789760739.932686:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5528a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.932699:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5528a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.932708:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5528a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.932715:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760739.932721:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5528a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.932731:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5528a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.932755:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5528a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.932763:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5528a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20893696->20897791] (req 20893696->20897791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760739.933057:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508333568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26851us (29444us total) trans 4294972421 rc 0/0 00000100:00100000:0.0:1789760739.933069:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5133 00000100:00100000:0.0:1789760739.933080:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718966790497 time 719010901184 state 0x2 ref 0 00000100:00100000:2.0:1789760739.939000:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760739.939069:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508333696 00000100:00100000:1.0:1789760739.939081:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508333696 00000100:00100000:1.0:1789760739.939322:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 718966790497 time 719010901184 state 0x0 ref 1 00000100:00100000:1.0:1789760739.939345:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719011161600 time 719010901184 state 0x1 ref 1 00000100:00100000:1.0:1789760739.939377:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719011161600 time 719044087662 state 0x0 ref 1 00000100:00100000:1.0:1789760739.939389:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508333696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760739.939437:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20897792, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:1.0:1789760739.939447:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55291 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.939463:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760739.939468:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760739.939473:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55291 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760739.939487:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55291 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760739.972886:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20897792 -> 20901888 00010000:00010000:2.0:1789760739.972906:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55291 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.972920:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55291 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.972930:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55291 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.972938:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760739.972943:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55291 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.972952:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55291 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.972973:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55291 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760739.972981:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55291 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20897792->20901887] (req 20897792->20901887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760739.973070:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508333696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33678us (34081us total) trans 4294972422 rc 0/0 00000100:00100000:2.0:1789760739.973079:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5134 00000100:00100000:2.0:1789760739.973099:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719011161600 time 719044087662 state 0x2 ref 0 00000001:00080000:0.0:1789760739.973760:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972422 is committed 00000100:00100000:2.0:1789760739.980137:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760739.980334:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508333824 00000100:00100000:0.0:1789760739.980346:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508333824 00000100:00100000:0.0:1789760739.980403:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719011161600 time 719044087662 state 0x0 ref 1 00000100:00100000:0.0:1789760739.980420:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719044348078 time 719044087662 state 0x1 ref 1 00000100:00100000:0.0:1789760739.980453:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719044348078 time 719085163048 state 0x0 ref 1 00000100:00100000:0.0:1789760739.980466:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508333824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760739.980589:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20901888, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:0.0:1789760739.980600:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55298 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.980618:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760739.980623:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760739.980631:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55298 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760739.980646:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55298 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760740.020077:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20901888 -> 20905984 00010000:00010000:0.0:1789760740.020113:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55298 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.020127:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55298 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.020137:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55298 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.020144:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.020159:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55298 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.020170:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55298 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.020195:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55298 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.020203:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55298 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20901888->20905983] (req 20901888->20905983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.020334:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508333824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39864us (40202us total) trans 4294972423 rc 0/0 00000100:00100000:0.0:1789760740.020348:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5135 00000100:00100000:0.0:1789760740.020359:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719044348078 time 719085163048 state 0x2 ref 0 00000001:00080000:2.0:1789760740.020677:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972423 is committed 00000100:00100000:3.0:1789760740.028371:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.028479:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508333952 00000100:00100000:0.0:1789760740.028492:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508333952 00000100:00100000:0.0:1789760740.028751:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719044348078 time 719085163048 state 0x0 ref 1 00000100:00100000:0.0:1789760740.028770:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719085423464 time 719085163048 state 0x1 ref 1 00000100:00100000:0.0:1789760740.028804:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719085423464 time 719133514378 state 0x0 ref 1 00000100:00100000:0.0:1789760740.028819:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508333952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760740.028877:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20905984, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:0.0:1789760740.028887:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5529f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.028908:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.028913:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.028921:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5529f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.028938:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5529f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.068154:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972424 is committed 00002000:00010000:3.0:1789760740.068244:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20905984 -> 20910080 00010000:00010000:3.0:1789760740.068262:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5529f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.068276:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5529f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.068786:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5529f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.068797:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760740.068802:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5529f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.068813:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5529f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.068836:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5529f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.068843:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5529f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20905984->20910079] (req 20905984->20910079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.069056:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508333952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40235us (40690us total) trans 4294972424 rc 0/0 00000100:00100000:2.0:1789760740.069067:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5136 00000100:00100000:2.0:1789760740.069078:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719085423464 time 719133514378 state 0x2 ref 0 00000100:00100000:3.0:1789760740.074747:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760740.074864:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508334080 00000100:00100000:2.0:1789760740.074875:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508334080 00000100:00100000:2.0:1789760740.074934:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719085423464 time 719133514378 state 0x0 ref 1 00000100:00100000:2.0:1789760740.074951:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719133774794 time 719133514378 state 0x1 ref 1 00000100:00100000:2.0:1789760740.075013:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719133774794 time 719179723019 state 0x0 ref 1 00000100:00100000:2.0:1789760740.075026:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508334080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760740.075097:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20910080, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:2.0:1789760740.075108:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe552a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.075126:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760740.075131:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.075138:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe552a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.075153:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe552a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.094671:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760740.102702:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972425 is committed 00002000:00010000:0.0:1789760740.102905:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20910080 -> 20914176 00010000:00010000:0.0:1789760740.102922:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe552a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.102935:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe552a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.102944:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe552a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.102952:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.102958:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe552a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.102967:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe552a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.102990:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe552a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.102999:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe552a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20910080->20914175] (req 20910080->20914175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.103107:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508334080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28076us (28364us total) trans 4294972425 rc 0/0 00000100:00100000:0.0:1789760740.103118:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5137 00000100:00100000:0.0:1789760740.103129:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719133774794 time 719179723019 state 0x2 ref 0 00000100:00100000:3.0:1789760740.110049:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.110545:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508334208 00000100:00100000:0.0:1789760740.110557:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508334208 00000100:00100000:0.0:1789760740.110609:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719133774794 time 719179723019 state 0x0 ref 1 00000100:00100000:0.0:1789760740.110627:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719179983435 time 719179723019 state 0x1 ref 1 00000100:00100000:0.0:1789760740.110657:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719179983435 time 719215367093 state 0x0 ref 1 00000100:00100000:0.0:1789760740.110669:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508334208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760740.110718:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20914176, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:0.0:1789760740.110729:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.110746:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.110751:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.110758:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.110774:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.139727:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972426 is committed 00002000:00010000:0.0:1789760740.139810:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20914176 -> 20918272 00010000:00010000:0.0:1789760740.139826:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.139839:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.139848:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.139855:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.139859:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.139868:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.139891:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.139899:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20914176->20918271] (req 20914176->20918271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.140000:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508334208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29327us (29956us total) trans 4294972426 rc 0/0 00000100:00100000:0.0:1789760740.140009:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5138 00000100:00100000:0.0:1789760740.140019:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719179983435 time 719215367093 state 0x2 ref 0 00000100:00100000:3.0:1789760740.147536:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760740.147637:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508334336 00000100:00100000:2.0:1789760740.147649:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508334336 00000100:00100000:2.0:1789760740.147710:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719179983435 time 719215367093 state 0x0 ref 1 00000100:00100000:2.0:1789760740.147727:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719215627509 time 719215367093 state 0x1 ref 1 00000100:00100000:2.0:1789760740.147791:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719215627509 time 719252501374 state 0x0 ref 1 00000100:00100000:2.0:1789760740.147804:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508334336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760740.147856:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20918272, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:2.0:1789760740.147865:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe552b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.147881:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760740.147885:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.147892:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe552b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.147907:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe552b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.195898:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972427 is committed 00002000:00010000:0.0:1789760740.196075:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20918272 -> 20922368 00010000:00010000:0.0:1789760740.196103:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe552b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.196121:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe552b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.196130:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe552b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.196136:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.196142:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe552b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.196151:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe552b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.196173:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe552b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.196180:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe552b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20918272->20922367] (req 20918272->20922367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.196365:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508334336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48556us (48832us total) trans 4294972427 rc 0/0 00000100:00100000:0.0:1789760740.196377:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5139 00000100:00100000:0.0:1789760740.196387:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719215627509 time 719252501374 state 0x2 ref 0 00000100:00100000:3.0:1789760740.199613:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.199677:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508334464 00000100:00100000:0.0:1789760740.199688:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508334464 00000100:00100000:0.0:1789760740.199746:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719215627509 time 719252501374 state 0x0 ref 1 00000100:00100000:0.0:1789760740.199762:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719252761790 time 719252501374 state 0x1 ref 1 00000100:00100000:0.0:1789760740.199795:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719252761790 time 719304505531 state 0x0 ref 1 00000100:00100000:0.0:1789760740.199808:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508334464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760740.199862:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20922368, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:0.0:1789760740.199872:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.199889:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.199894:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.199900:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.199916:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.229874:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972428 is committed 00002000:00010000:0.0:1789760740.229955:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20922368 -> 20926464 00010000:00010000:0.0:1789760740.229971:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.229984:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.230071:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.230079:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.230093:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.230105:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.230128:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.230136:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20922368->20926463] (req 20922368->20926463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.230263:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508334464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30451us (30654us total) trans 4294972428 rc 0/0 00000100:00100000:0.0:1789760740.230274:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5140 00000100:00100000:0.0:1789760740.230316:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719252761790 time 719304505531 state 0x2 ref 0 00000100:00100000:2.0:1789760740.235245:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.235332:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508334592 00000100:00100000:0.0:1789760740.235343:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508334592 00000100:00100000:0.0:1789760740.235409:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719252761790 time 719304505531 state 0x0 ref 1 00000100:00100000:0.0:1789760740.235425:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719304765947 time 719304505531 state 0x1 ref 1 00000100:00100000:0.0:1789760740.235458:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719304765947 time 719340168456 state 0x0 ref 1 00000100:00100000:0.0:1789760740.235471:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508334592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760740.235527:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20926464, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:0.0:1789760740.235537:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.235554:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.235558:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.235566:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.235581:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.262517:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972429 is committed 00002000:00010000:0.0:1789760740.262583:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20926464 -> 20930560 00010000:00010000:0.0:1789760740.262599:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.262612:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.262619:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.262623:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.262627:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.262634:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.262653:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.262657:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20926464->20930559] (req 20926464->20930559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.262792:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508334592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27315us (27553us total) trans 4294972429 rc 0/0 00000100:00100000:0.0:1789760740.262803:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5141 00000100:00100000:0.0:1789760740.262814:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719304765947 time 719340168456 state 0x2 ref 0 00000100:00100000:3.0:1789760740.266517:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.266625:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508334720 00000100:00100000:0.0:1789760740.266636:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508334720 00000100:00100000:0.0:1789760740.266702:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719304765947 time 719340168456 state 0x0 ref 1 00000100:00100000:0.0:1789760740.266718:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719340428872 time 719340168456 state 0x1 ref 1 00000100:00100000:0.0:1789760740.266738:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719340428872 time 719371448972 state 0x0 ref 1 00000100:00100000:0.0:1789760740.266750:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508334720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760740.266801:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20930560, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:0.0:1789760740.266812:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.266830:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.266835:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.266842:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.266857:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.294130:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972430 is committed 00002000:00010000:0.0:1789760740.294218:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20930560 -> 20934656 00010000:00010000:0.0:1789760740.294233:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.294243:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.294249:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.294254:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.294258:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.294265:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.294309:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.294318:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20930560->20934655] (req 20930560->20934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.294448:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508334720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27693us (27935us total) trans 4294972430 rc 0/0 00000100:00100000:2.0:1789760740.294460:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5142 00000100:00100000:2.0:1789760740.294470:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719340428872 time 719371448972 state 0x2 ref 0 00000100:00100000:3.0:1789760740.305773:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.306079:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508334848 00000100:00100000:0.0:1789760740.306100:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508334848 00000100:00100000:0.0:1789760740.306160:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719340428872 time 719371448972 state 0x0 ref 1 00000100:00100000:0.0:1789760740.306176:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719371709388 time 719371448972 state 0x1 ref 1 00000100:00100000:0.0:1789760740.306206:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719371709388 time 719410917192 state 0x0 ref 1 00000100:00100000:0.0:1789760740.306218:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508334848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760740.307668:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20934656, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:0.0:1789760740.307681:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.307698:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.307704:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.307712:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.307727:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.344558:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972431 is committed 00002000:00010000:0.0:1789760740.344638:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20934656 -> 20938752 00010000:00010000:0.0:1789760740.344654:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.344665:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.344674:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.344681:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.344686:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.344695:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.344717:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.344778:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe552d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20934656->20938751] (req 20934656->20938751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.345054:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508334848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38832us (39286us total) trans 4294972431 rc 0/0 00000100:00100000:0.0:1789760740.345069:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5143 00000100:00100000:0.0:1789760740.345080:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719371709388 time 719410917192 state 0x2 ref 0 00000100:00100000:3.0:1789760740.348408:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.348451:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508334976 00000100:00100000:0.0:1789760740.348462:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508334976 00000100:00100000:0.0:1789760740.348519:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719371709388 time 719410917192 state 0x0 ref 1 00000100:00100000:0.0:1789760740.348536:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719411177608 time 719410917192 state 0x1 ref 1 00000100:00100000:0.0:1789760740.348569:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719411177608 time 719453279626 state 0x0 ref 1 00000100:00100000:0.0:1789760740.348582:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508334976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760740.348640:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20938752, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:0.0:1789760740.348650:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.348667:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.348671:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.348678:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.348695:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.372916:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972432 is committed 00002000:00010000:0.0:1789760740.372994:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20938752 -> 20942848 00010000:00010000:0.0:1789760740.373073:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.373098:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.373108:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.373117:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.373122:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.373132:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.373157:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.373227:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe552d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20938752->20942847] (req 20938752->20942847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.373519:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508334976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24933us (25111us total) trans 4294972432 rc 0/0 00000100:00100000:0.0:1789760740.373530:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5144 00000100:00100000:0.0:1789760740.373541:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719411177608 time 719453279626 state 0x2 ref 0 00000100:00100000:3.0:1789760740.378911:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760740.379018:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508335104 00000100:00100000:2.0:1789760740.379028:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508335104 00000100:00100000:2.0:1789760740.379096:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719411177608 time 719453279626 state 0x0 ref 1 00000100:00100000:2.0:1789760740.379113:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719453540042 time 719453279626 state 0x1 ref 1 00000100:00100000:2.0:1789760740.379487:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719453540042 time 719484194940 state 0x0 ref 1 00000100:00100000:2.0:1789760740.379502:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508335104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760740.379555:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20942848, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:2.0:1789760740.379566:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe552de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.379584:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760740.379589:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.379595:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe552de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.379616:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe552de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.412220:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972433 is committed 00002000:00010000:0.0:1789760740.412361:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20942848 -> 20946944 00010000:00010000:0.0:1789760740.412378:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe552de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.412390:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe552de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.412507:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe552de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.412515:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.412520:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe552de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.412530:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe552de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.412554:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe552de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.412563:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe552de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20942848->20946943] (req 20942848->20946943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.412656:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508335104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33152us (33751us total) trans 4294972433 rc 0/0 00000100:00100000:0.0:1789760740.412666:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5145 00000100:00100000:0.0:1789760740.412676:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719453540042 time 719484194940 state 0x2 ref 0 00000100:00100000:3.0:1789760740.416680:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760740.416717:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.416809:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508335360 00000100:00100000:0.0:1789760740.416821:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508335360 00000100:00100000:2.0:1789760740.416845:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508335616 00000100:00100000:2.0:1789760740.416852:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508335616 00000100:00100000:0.0:1789760740.416996:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 889 00000100:00100000:0.0:1789760740.417008:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2409:x1876699508335360:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760740.417064:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 890 00000100:00100000:2.0:1789760740.417073:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508335616:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760740.417156:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2409:x1876699508335360:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 90us (425us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760740.417166:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 889 00000100:00100000:2.0:1789760740.417741:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508335616:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 667us (1022us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760740.417749:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 890 00000100:00080000:3.0:1789760740.418033:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760740.419361:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.419367:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760740.419402:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760740.419432:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.419475:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508335488 00000100:00100000:0.0:1789760740.419484:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508335488 00000100:00100000:0.0:1789760740.419540:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508335744 00000100:00100000:0.0:1789760740.419543:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508335744 00000100:00100000:0.0:1789760740.419567:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 891 00000100:00100000:0.0:1789760740.419576:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508335488:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760740.419631:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508335488:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 56us (236us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760740.419638:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 891 00000100:00100000:0.0:1789760740.419651:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 892 00000100:00100000:0.0:1789760740.419657:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508335744:12345-192.168.203.23@tcp:400:kworker.0 00000100:00080000:3.0:1789760740.419673:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760740.419682:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508335744:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 25us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760740.419689:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 892 00000100:00100000:1.0:1789760740.419732:0:6460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508335232 00000100:00100000:1.0:1789760740.419738:0:6460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508335232 00000100:00100000:1.0:1789760740.419783:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 471 00000100:00100000:1.0:1789760740.419792:0:6460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cafc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2409:x1876699508335232:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760740.419837:0:6460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cafc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2409:x1876699508335232:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 46us (482us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760740.419844:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 471 00000100:00100000:2.0:1789760740.419875:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508335872 00000100:00100000:2.0:1789760740.419881:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508335872 00000100:00100000:2.0:1789760740.419919:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719453540042 time 719484194940 state 0x0 ref 1 00000100:00100000:2.0:1789760740.419932:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719484455356 time 719484194940 state 0x1 ref 1 00000100:00100000:2.0:1789760740.419961:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719484455356 time 719524670948 state 0x0 ref 1 00000100:00100000:2.0:1789760740.419972:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508335872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760740.420093:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20946944, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:2.0:1789760740.420104:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe552e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.420120:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760740.420125:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.420131:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe552e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.420147:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe552e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.448437:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972434 is committed 00002000:00010000:0.0:1789760740.448475:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20946944 -> 20951040 00010000:00010000:0.0:1789760740.448491:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe552e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.448508:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe552e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.448516:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe552e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.448523:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.448528:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe552e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.448537:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe552e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.448560:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe552e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.448568:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe552e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20946944->20951039] (req 20946944->20951039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.448695:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508335872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28719us (29326us total) trans 4294972434 rc 0/0 00000100:00100000:0.0:1789760740.448707:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5146 00000100:00100000:0.0:1789760740.448718:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719484455356 time 719524670948 state 0x2 ref 0 00000100:00100000:0.0:1789760740.457011:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760740.457069:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508336000 00000100:00100000:3.0:1789760740.457092:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508336000 00000100:00100000:3.0:1789760740.457160:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719484455356 time 719524670948 state 0x0 ref 1 00000100:00100000:3.0:1789760740.457178:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719524931364 time 719524670948 state 0x1 ref 1 00000100:00100000:3.0:1789760740.457214:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719524931364 time 719561923739 state 0x0 ref 1 00000100:00100000:3.0:1789760740.457228:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508336000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760740.457277:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20951040, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:3.0:1789760740.457326:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe552ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.457343:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760740.457347:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.457359:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe552ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.457375:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe552ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760740.486882:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972435 is committed 00002000:00010000:0.0:1789760740.486984:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20951040 -> 20955136 00010000:00010000:0.0:1789760740.487005:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe552ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.487018:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe552ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.487025:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe552ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.487030:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.487033:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe552ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.487040:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe552ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.487059:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe552ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.487064:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe552ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20951040->20955135] (req 20951040->20955135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.487162:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508336000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29932us (30155us total) trans 4294972435 rc 0/0 00000100:00100000:0.0:1789760740.487173:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5147 00000100:00100000:0.0:1789760740.487184:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719524931364 time 719561923739 state 0x2 ref 0 00000100:00100000:3.0:1789760740.491744:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.491878:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508336128 00000100:00100000:0.0:1789760740.491892:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508336128 00000100:00100000:0.0:1789760740.491957:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719524931364 time 719561923739 state 0x0 ref 1 00000100:00100000:0.0:1789760740.491973:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719562184155 time 719561923739 state 0x1 ref 1 00000100:00100000:0.0:1789760740.492010:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719562184155 time 719596710591 state 0x0 ref 1 00000100:00100000:0.0:1789760740.492023:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508336128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760740.492098:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20955136, mtime: 0x6aad94e4, atime: 0x0, ctime: 0x6aad94e4, blocks: 0xa007 00010000:00010000:0.0:1789760740.492109:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe552f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.492127:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.492132:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.492139:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe552f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.492155:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe552f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.531240:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972436 is committed 00002000:00010000:2.0:1789760740.531511:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20955136 -> 20959232 00002000:00010000:2.0:1789760740.531516:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760740 -> 1789760741 00002000:00010000:2.0:1789760740.531519:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760740 -> 1789760741 00010000:00010000:2.0:1789760740.531533:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe552f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.531546:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe552f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.531555:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe552f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.531658:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760740.531664:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe552f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.531678:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe552f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.531702:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe552f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.531711:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe552f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20955136->20959231] (req 20955136->20959231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.531822:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508336128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39796us (40091us total) trans 4294972436 rc 0/0 00000100:00100000:2.0:1789760740.531832:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5148 00000100:00100000:2.0:1789760740.532102:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719562184155 time 719596710591 state 0x2 ref 0 00000100:00100000:3.0:1789760740.543644:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.543738:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508336256 00000100:00100000:0.0:1789760740.543748:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508336256 00000100:00100000:0.0:1789760740.543803:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719562184155 time 719596710591 state 0x0 ref 1 00000100:00100000:0.0:1789760740.543818:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719596971007 time 719596710591 state 0x1 ref 1 00000100:00100000:0.0:1789760740.543853:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719596971007 time 719648563464 state 0x0 ref 1 00000100:00100000:0.0:1789760740.543864:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508336256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760740.543915:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20959232, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa007 00010000:00010000:0.0:1789760740.543924:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe552fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.543941:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.543946:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.543952:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe552fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.543968:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe552fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760740.586193:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20959232 -> 20963328 00010000:00010000:0.0:1789760740.586215:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe552fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.586228:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe552fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.586237:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe552fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.586244:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.586250:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe552fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.586259:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe552fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.586282:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe552fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.586310:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe552fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20959232->20963327] (req 20959232->20963327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.586413:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508336256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42544us (42771us total) trans 4294972437 rc 0/0 00000100:00100000:0.0:1789760740.586424:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5149 00000100:00100000:0.0:1789760740.586446:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719596971007 time 719648563464 state 0x2 ref 0 00000001:00080000:1.0:1789760740.586657:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972437 is committed 00000100:00100000:0.0:1789760740.592445:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760740.592718:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508336384 00000100:00100000:3.0:1789760740.592729:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508336384 00000100:00100000:3.0:1789760740.592790:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719596971007 time 719648563464 state 0x0 ref 1 00000100:00100000:3.0:1789760740.592806:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719648823880 time 719648563464 state 0x1 ref 1 00000100:00100000:3.0:1789760740.592839:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719648823880 time 719697549072 state 0x0 ref 1 00000100:00100000:3.0:1789760740.592853:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508336384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760740.592905:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20963328, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa007 00010000:00010000:3.0:1789760740.592915:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55301 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.592934:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760740.592939:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.592947:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55301 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.592962:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55301 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760740.640651:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972438 is committed 00002000:00010000:0.0:1789760740.642495:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20963328 -> 20967424 00010000:00010000:0.0:1789760740.642517:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55301 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.642531:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55301 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.642541:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55301 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.642554:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.642560:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55301 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.642570:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55301 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.642593:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55301 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.642600:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55301 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20963328->20967423] (req 20963328->20967423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.644540:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508336384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49852us (50277us total) trans 4294972438 rc 0/0 00000100:00100000:0.0:1789760740.644555:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5150 00000100:00100000:0.0:1789760740.644567:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719648823880 time 719697549072 state 0x2 ref 0 00000100:00100000:3.0:1789760740.648043:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.648091:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508336512 00000100:00100000:0.0:1789760740.648102:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508336512 00000100:00100000:0.0:1789760740.648160:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719648823880 time 719697549072 state 0x0 ref 1 00000100:00100000:0.0:1789760740.648177:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719697809488 time 719697549072 state 0x1 ref 1 00000100:00100000:0.0:1789760740.648204:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719697809488 time 719752914828 state 0x0 ref 1 00000100:00100000:0.0:1789760740.648220:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508336512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760740.648269:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20967424, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa007 00010000:00010000:0.0:1789760740.648278:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55308 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.648380:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.648385:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.648393:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55308 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.648409:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55308 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.696108:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972439 is committed 00002000:00010000:0.0:1789760740.696321:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20967424 -> 20971520 00010000:00010000:0.0:1789760740.696340:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55308 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.696352:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55308 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.696362:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55308 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.696370:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.696375:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55308 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.696385:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55308 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.696409:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55308 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.696417:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55308 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20967424->20971519] (req 20967424->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.696517:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508336512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48294us (48478us total) trans 4294972439 rc 0/0 00000100:00100000:0.0:1789760740.696526:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5151 00000100:00100000:0.0:1789760740.696537:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719697809488 time 719752914828 state 0x2 ref 0 00000100:00100000:0.0:1789760740.701645:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760740.702026:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508336640 00000100:00100000:2.0:1789760740.702039:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508336640 00000100:00100000:2.0:1789760740.702113:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719697809488 time 719752914828 state 0x0 ref 1 00000100:00100000:2.0:1789760740.702130:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719753175244 time 719752914828 state 0x1 ref 1 00000100:00100000:2.0:1789760740.702162:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719753175244 time 719806872721 state 0x0 ref 1 00000100:00100000:2.0:1789760740.702174:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508336640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760740.702223:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20971520, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa007 00010000:00010000:2.0:1789760740.702232:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5530f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.702251:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760740.702255:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760740.702262:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5530f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.702277:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5530f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760740.765362:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20971520 -> 20975616 00002000:00010000:0.0:1789760740.765507:0:14639:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:0.0:1789760740.765523:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5530f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.765536:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5530f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.765545:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5530f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.765556:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.765560:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5530f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.765569:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5530f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.765591:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5530f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.765598:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5530f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20971520->20975615] (req 20971520->20975615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.765703:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508336640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 63524us (64060us total) trans 4294972440 rc 0/0 00000100:00100000:0.0:1789760740.765713:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5152 00000100:00100000:0.0:1789760740.765724:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719753175244 time 719806872721 state 0x2 ref 0 00000001:00080000:3.0:1789760740.769999:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972440 is committed 00000100:00100000:2.0:1789760740.770252:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.770768:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508336768 00000100:00100000:0.0:1789760740.770778:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508336768 00000100:00100000:0.0:1789760740.770838:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719753175244 time 719806872721 state 0x0 ref 1 00000100:00100000:0.0:1789760740.770855:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719807133137 time 719806872721 state 0x1 ref 1 00000100:00100000:0.0:1789760740.770922:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719807133137 time 719875630828 state 0x0 ref 1 00000100:00100000:0.0:1789760740.770938:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508336768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760740.770989:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20975616, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:0.0:1789760740.770997:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55316 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.771014:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.771019:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.771026:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55316 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.771040:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55316 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760740.817548:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20975616 -> 20979712 00010000:00010000:0.0:1789760740.817570:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55316 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.817584:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55316 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.817594:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55316 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.817601:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.817606:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55316 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.817617:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55316 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.817637:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55316 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.817642:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55316 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20975616->20979711] (req 20975616->20979711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.817731:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508336768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46780us (47468us total) trans 4294972441 rc 0/0 00000100:00100000:0.0:1789760740.817741:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5153 00000100:00100000:0.0:1789760740.817751:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719807133137 time 719875630828 state 0x2 ref 0 00000001:00080000:0.0:1789760740.821376:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972441 is committed 00000100:00100000:2.0:1789760740.822854:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.823214:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508336896 00000100:00100000:0.0:1789760740.823224:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508336896 00000100:00100000:0.0:1789760740.823280:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719807133137 time 719875630828 state 0x0 ref 1 00000100:00100000:0.0:1789760740.823339:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719875891244 time 719875630828 state 0x1 ref 1 00000100:00100000:1.0:1789760740.823407:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719875891244 time 719928113664 state 0x0 ref 1 00000100:00100000:1.0:1789760740.823423:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508336896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760740.823474:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20979712, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:1.0:1789760740.823483:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5531d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.823499:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760740.823504:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760740.823511:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5531d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760740.823526:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5531d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.855932:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972442 is committed 00002000:00010000:2.0:1789760740.856245:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20979712 -> 20983808 00010000:00010000:2.0:1789760740.856261:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5531d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.856273:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5531d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.856282:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5531d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.856311:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760740.856316:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5531d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.856325:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5531d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.856351:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5531d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760740.856359:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5531d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20979712->20983807] (req 20979712->20983807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760740.856455:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508336896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33031us (33605us total) trans 4294972442 rc 0/0 00000100:00100000:2.0:1789760740.856466:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5154 00000100:00100000:2.0:1789760740.856477:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719875891244 time 719928113664 state 0x2 ref 0 00000100:00100000:1.0:1789760740.861701:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760740.861791:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508337024 00000100:00100000:0.0:1789760740.861801:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508337024 00000100:00100000:0.0:1789760740.861872:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719875891244 time 719928113664 state 0x0 ref 1 00000100:00100000:0.0:1789760740.861889:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719928374080 time 719928113664 state 0x1 ref 1 00000100:00100000:0.0:1789760740.861916:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719928374080 time 719966626316 state 0x0 ref 1 00000100:00100000:0.0:1789760740.861928:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508337024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760740.862114:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20983808, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:0.0:1789760740.862126:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55324 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.862141:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760740.862145:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760740.862153:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55324 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.862169:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55324 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760740.916069:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972443 is committed 00002000:00010000:0.0:1789760740.916155:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20983808 -> 20987904 00010000:00010000:0.0:1789760740.916172:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55324 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.916185:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55324 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.916195:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55324 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.916210:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.916215:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55324 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.916226:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55324 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.916250:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55324 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.916259:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55324 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20983808->20987903] (req 20983808->20987903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.916388:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508337024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54455us (54690us total) trans 4294972443 rc 0/0 00000100:00100000:0.0:1789760740.916399:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5155 00000100:00100000:0.0:1789760740.916409:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719928374080 time 719966626316 state 0x2 ref 0 00000100:00100000:0.0:1789760740.919912:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760740.920139:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508337152 00000100:00100000:3.0:1789760740.920149:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508337152 00000100:00100000:3.0:1789760740.920210:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719928374080 time 719966626316 state 0x0 ref 1 00000100:00100000:3.0:1789760740.920228:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719966886732 time 719966626316 state 0x1 ref 1 00000100:00100000:3.0:1789760740.920260:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719966886732 time 720024970346 state 0x0 ref 1 00000100:00100000:3.0:1789760740.920273:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508337152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760740.920369:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20987904, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:3.0:1789760740.920381:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5532b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.920399:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760740.920404:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760740.920412:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5532b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760740.920429:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5532b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760740.990693:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972444 is committed 00002000:00010000:0.0:1789760740.990815:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20987904 -> 20992000 00010000:00010000:0.0:1789760740.990832:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5532b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.990844:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5532b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.990858:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5532b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.990866:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760740.990870:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5532b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.990878:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5532b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.990900:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5532b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760740.990908:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5532b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20987904->20991999] (req 20987904->20991999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760740.991002:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508337152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 70725us (71095us total) trans 4294972444 rc 0/0 00000100:00100000:0.0:1789760740.991012:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5156 00000100:00100000:0.0:1789760740.991022:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719966886732 time 720024970346 state 0x2 ref 0 00000100:00100000:0.0:1789760741.003203:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760741.003325:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508337280 00000100:00100000:2.0:1789760741.003338:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508337280 00000100:00100000:2.0:1789760741.003424:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 719966886732 time 720024970346 state 0x0 ref 1 00000100:00100000:2.0:1789760741.003439:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720025230762 time 720024970346 state 0x1 ref 1 00000100:00100000:2.0:1789760741.003469:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720025230762 time 720108179860 state 0x0 ref 1 00000100:00100000:2.0:1789760741.003482:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508337280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760741.003532:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20992000, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:2.0:1789760741.003542:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55332 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.003560:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.003565:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.003571:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55332 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.003586:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55332 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760741.004092:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760741.039478:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20992000 -> 20996096 00010000:00010000:2.0:1789760741.039503:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55332 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.039517:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55332 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.039527:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55332 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.039535:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760741.039540:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55332 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.039550:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55332 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.039575:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55332 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.039583:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55332 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20992000->20996095] (req 20992000->20996095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.039704:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972445 is committed 00000100:00100000:2.0:1789760741.039836:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508337280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36349us (36639us total) trans 4294972445 rc 0/0 00000100:00100000:2.0:1789760741.039850:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5157 00000100:00100000:2.0:1789760741.039861:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720025230762 time 720108179860 state 0x2 ref 0 00000100:00100000:3.0:1789760741.043728:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760741.044888:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508337408 00000100:00100000:0.0:1789760741.044902:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508337408 00000100:00100000:0.0:1789760741.044965:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720025230762 time 720108179860 state 0x0 ref 1 00000100:00100000:0.0:1789760741.044986:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720108440276 time 720108179860 state 0x1 ref 1 00000100:00100000:0.0:1789760741.045026:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720108440276 time 720149735083 state 0x0 ref 1 00000100:00100000:0.0:1789760741.045041:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d45450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508337408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760741.045106:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 20996096, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:0.0:1789760741.045117:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55339 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.045135:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.045140:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.045147:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55339 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.045163:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55339 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.087421:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972446 is committed 00002000:00010000:0.0:1789760741.089222:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 20996096 -> 21000192 00010000:00010000:0.0:1789760741.089244:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55339 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.089258:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55339 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.089267:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55339 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.089278:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760741.089306:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55339 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.089315:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55339 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.089337:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55339 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.089345:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55339 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [20996096->21000191] (req 20996096->21000191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.089456:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d45450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508337408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44414us (45734us total) trans 4294972446 rc 0/0 00000100:00100000:0.0:1789760741.089467:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5158 00000100:00100000:0.0:1789760741.089477:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720108440276 time 720149735083 state 0x2 ref 0 00000100:00100000:1.0:1789760741.092855:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760741.092944:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508337536 00000100:00100000:0.0:1789760741.092956:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508337536 00000100:00100000:0.0:1789760741.093011:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720108440276 time 720149735083 state 0x0 ref 1 00000100:00100000:0.0:1789760741.093027:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720149995499 time 720149735083 state 0x1 ref 1 00000100:00100000:0.0:1789760741.093070:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720149995499 time 720197779750 state 0x0 ref 1 00000100:00100000:0.0:1789760741.093097:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508337536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760741.093150:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21000192, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:0.0:1789760741.093160:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55340 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.093180:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.093183:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.093190:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55340 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.093204:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55340 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.118670:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760741.122717:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972447 is committed 00002000:00010000:1.0:1789760741.122829:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21000192 -> 21004288 00010000:00010000:1.0:1789760741.122847:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55340 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.122861:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55340 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.122871:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55340 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.122878:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760741.122883:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55340 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.122892:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55340 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.122916:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55340 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.122924:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55340 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21000192->21004287] (req 21000192->21004287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.123023:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508337536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29923us (30172us total) trans 4294972447 rc 0/0 00000100:00100000:1.0:1789760741.123033:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5159 00000100:00100000:1.0:1789760741.123044:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720149995499 time 720197779750 state 0x2 ref 0 00000100:00100000:2.0:1789760741.134464:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760741.134552:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508337664 00000100:00100000:1.0:1789760741.134564:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508337664 00000100:00100000:1.0:1789760741.134624:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720149995499 time 720197779750 state 0x0 ref 1 00000100:00100000:1.0:1789760741.134639:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720198040166 time 720197779750 state 0x1 ref 1 00000100:00100000:1.0:1789760741.134671:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720198040166 time 720239381540 state 0x0 ref 1 00000100:00100000:1.0:1789760741.134685:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508337664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760741.134737:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21004288, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:1.0:1789760741.134747:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55347 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.134765:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760741.134769:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.134777:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55347 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.134792:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55347 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760741.180915:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972448 is committed 00002000:00010000:3.0:1789760741.181012:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21004288 -> 21008384 00010000:00010000:3.0:1789760741.181028:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55347 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.181042:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55347 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.181051:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55347 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.181057:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760741.181062:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55347 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.181071:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55347 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.181269:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55347 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.181279:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55347 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21004288->21008383] (req 21004288->21008383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.181412:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508337664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46723us (46960us total) trans 4294972448 rc 0/0 00000100:00100000:3.0:1789760741.181424:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5160 00000100:00100000:3.0:1789760741.181435:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720198040166 time 720239381540 state 0x2 ref 0 00000100:00100000:3.0:1789760741.186934:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760741.187020:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508337792 00000100:00100000:0.0:1789760741.187032:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508337792 00000100:00100000:0.0:1789760741.187101:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720198040166 time 720239381540 state 0x0 ref 1 00000100:00100000:0.0:1789760741.187119:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720239641956 time 720239381540 state 0x1 ref 1 00000100:00100000:0.0:1789760741.187154:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720239641956 time 720291864495 state 0x0 ref 1 00000100:00100000:0.0:1789760741.187168:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508337792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760741.187217:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21008384, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:0.0:1789760741.187229:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5534e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.187247:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.187251:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.187258:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5534e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.187274:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5534e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.234253:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972449 is committed 00002000:00010000:3.0:1789760741.234354:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21008384 -> 21012480 00010000:00010000:3.0:1789760741.234371:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5534e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.234384:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5534e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.234393:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5534e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.234402:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760741.234407:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5534e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.234417:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5534e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.234503:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5534e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.234511:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5534e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21008384->21012479] (req 21008384->21012479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.234645:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508337792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47474us (47715us total) trans 4294972449 rc 0/0 00000100:00100000:3.0:1789760741.234655:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5161 00000100:00100000:3.0:1789760741.234666:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720239641956 time 720291864495 state 0x2 ref 0 00000100:00100000:1.0:1789760741.242526:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760741.245385:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508337920 00000100:00100000:3.0:1789760741.245401:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508337920 00000100:00100000:3.0:1789760741.245539:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720239641956 time 720291864495 state 0x0 ref 1 00000100:00100000:3.0:1789760741.245556:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720292124911 time 720291864495 state 0x1 ref 1 00000100:00100000:3.0:1789760741.245642:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720292124911 time 720350351544 state 0x0 ref 1 00000100:00100000:3.0:1789760741.245656:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508337920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760741.245709:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21012480, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:3.0:1789760741.245718:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55355 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.245736:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760741.245741:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.245748:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55355 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.245762:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55355 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.246615:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534068864:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760741.246683:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760741.247010:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04985468380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534068992:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760741.247039:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760741.247073:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534068864 00000100:00100000:3.0:1789760741.248914:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534068992 00000100:00100000:3.0:1789760741.248924:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534068992 00000100:00100000:3.0:1789760741.248985:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 368 00000100:00100000:3.0:1789760741.248996:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534068992:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760741.249032:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760741.249136:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534068992:12345-0@lo:13:osp-pre-1-0.0 Request processed in 143us (2095us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760741.249144:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 368 00000100:00100000:3.0:1789760741.249171:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534069120:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760741.249207:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.249382:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534069504:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.249410:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760741.249948:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534068864 00000100:00100000:2.0:1789760741.249980:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534069120 00000100:00100000:2.0:1789760741.249983:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534069120 00000100:00100000:2.0:1789760741.249999:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 369 00000100:00100000:2.0:1789760741.250007:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534068864:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789760741.250022:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760741.250105:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534068864:12345-0@lo:13:osp-pre-1-1.0 Request processed in 98us (3420us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.250113:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 369 00000100:00100000:2.0:1789760741.250128:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:2.0:1789760741.250135:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534069120:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760741.250145:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3836739584 avail 3828350976 00000100:00100000:2.0:1789760741.250180:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534069120:12345-0@lo:13:osp-pre-0-1.0 Request processed in 41us (967us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.250188:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00100000:3.0:1789760741.250336:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534070016:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.250365:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.250398:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049be26c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534070528:0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.250408:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534069504 00000100:00100000:2.0:1789760741.250414:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534069504 00000100:00100000:2.0:1789760741.250443:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534070016 00000100:00100000:2.0:1789760741.250447:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534070016 00000100:00100000:2.0:1789760741.250468:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 893 00000100:00100000:2.0:1789760741.250475:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534069504:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.250724:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534069504:12345-0@lo:400:kworker.0 Request processed in 249us (1313us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.250731:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 893 00000100:00100000:2.0:1789760741.250745:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 894 00000100:00100000:2.0:1789760741.250751:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534070016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.250782:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534070016:12345-0@lo:400:kworker.0 Request processed in 31us (416us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.250788:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 894 00000100:00080000:2.0:1789760741.250807:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760741.250812:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760741.250814:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760741.253144:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760741.253200:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534070528 00000100:00100000:2.0:1789760741.253210:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534070528 00000100:00100000:2.0:1789760741.253264:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 895 00000100:00100000:2.0:1789760741.253274:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534070528:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.253757:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534069248:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760741.253817:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.253859:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534069760:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.253889:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.253910:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534070272:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.253931:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.253954:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1789760741.253961:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3472:1876699534069376:0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.254124:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534070528:12345-0@lo:400:kworker.0 Request processed in 850us (3700us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.254134:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 895 00000100:00100000:2.0:1789760741.254157:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534069760 00000100:00100000:2.0:1789760741.254162:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534069760 00000100:00100000:2.0:1789760741.254195:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 896 00000100:00100000:2.0:1789760741.254203:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534069760:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.254240:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534069760:12345-0@lo:400:kworker.0 Request processed in 39us (350us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.254247:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 896 00000100:00080000:2.0:1789760741.254267:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760741.254348:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.254377:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534069888:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.254403:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.254433:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049be26e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3472:1876699534070400:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.254456:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.254495:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534069760:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.254533:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534070400 00000100:00100000:3.0:1789760741.254537:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534070400 00000100:00100000:3.0:1789760741.254573:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 897 00000100:00100000:3.0:1789760741.254580:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3472:x1876699534070400:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.254635:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3472:x1876699534070400:12345-0@lo:400:kworker.0 Request processed in 55us (177us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760741.254642:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 897 00000100:00100000:2.0:1789760741.254649:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534069248 00000100:00100000:2.0:1789760741.254654:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534069248 00000100:00100000:2.0:1789760741.254689:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 371 00000100:00100000:2.0:1789760741.254698:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534069248:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760741.254715:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3836739584 avail 3828350976 00000100:00100000:2.0:1789760741.254756:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534069248:12345-0@lo:13:osp-pre-0-0.0 Request processed in 60us (938us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.254763:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 371 00000100:00100000:2.0:1789760741.254809:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534068864:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760741.254843:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534069632:0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.254938:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760741.254965:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3475:1876699534070144:0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.254988:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760741.254997:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534070272 00000100:00100000:3.0:1789760741.255003:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534070272 00000100:00100000:2.0:1789760741.255021:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534069120:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760741.255038:0:8102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:2.0:1789760741.255047:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534069888 00000100:00100000:3.0:1789760741.255048:0:8102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534070272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.255053:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534069888 00000100:00100000:2.0:1789760741.255091:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534069632 00000100:00100000:2.0:1789760741.255094:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534069632 00000100:00100000:3.0:1789760741.255101:0:8102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534070272:12345-0@lo:400:kworker.0 Request processed in 55us (1167us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.255110:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 430 00000100:00100000:3.0:1789760741.255111:0:8102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00100000:2.0:1789760741.255120:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534069888:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.255135:0:6459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534069376 00000100:00100000:3.0:1789760741.255141:0:6459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534069376 00000100:00100000:2.0:1789760741.255159:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534069888:12345-0@lo:400:kworker.0 Request processed in 40us (753us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.255167:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 430 00000100:00100000:3.0:1789760741.255171:0:6459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00100000:3.0:1789760741.255179:0:6459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cad450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3472:x1876699534069376:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.255183:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 431 00000100:00100000:2.0:1789760741.255192:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534069632:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.255222:0:6459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cad450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3472:x1876699534069376:12345-0@lo:400:kworker.0 Request processed in 43us (873us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760741.255230:0:6459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00100000:2.0:1789760741.255235:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534069632:12345-0@lo:400:kworker.0 Request processed in 43us (296us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.255242:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 431 00000100:00100000:2.0:1789760741.255262:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534070144 00000100:00100000:2.0:1789760741.255265:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534070144 00000100:00080000:3.0:1789760741.255266:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760741.255271:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760741.255274:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789760741.255363:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 898 00000100:00100000:2.0:1789760741.255371:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3475:x1876699534070144:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.255398:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3475:x1876699534070144:12345-0@lo:400:kworker.0 Request processed in 29us (410us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760741.255403:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 898 00000100:00100000:2.0:1789760741.256456:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534069632:0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.256469:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3475:1876699534070144:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.256681:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3472:1876699534069376:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.256693:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534069888:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.256703:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049be26e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3472:1876699534070400:0@lo:400:kworker.0 00000100:00100000:3.0:1789760741.256722:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534069248:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760741.256732:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534070272:0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.256881:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049be26c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534070528:0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.256901:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04985468380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534068992:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760741.256922:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534069504:0@lo:400:kworker.0 00000100:00100000:2.0:1789760741.256931:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534070016:0@lo:400:kworker.0 00000001:00080000:1.0:1789760741.285927:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972450 is committed 00002000:00010000:2.0:1789760741.289740:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21012480 -> 21016576 00010000:00010000:2.0:1789760741.289763:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55355 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.289777:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55355 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.289791:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55355 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.289798:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760741.289803:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55355 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.289813:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55355 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.289838:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55355 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.289845:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55355 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21012480->21016575] (req 21012480->21016575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.289954:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508337920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44296us (47433us total) trans 4294972450 rc 0/0 00000100:00100000:2.0:1789760741.289966:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5162 00000100:00100000:2.0:1789760741.289976:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720292124911 time 720350351544 state 0x2 ref 0 00000100:00100000:0.0:1789760741.294981:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760741.295098:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508338048 00000100:00100000:3.0:1789760741.295110:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508338048 00000100:00100000:3.0:1789760741.295170:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720292124911 time 720350351544 state 0x0 ref 1 00000100:00100000:3.0:1789760741.295186:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720350611960 time 720350351544 state 0x1 ref 1 00000100:00100000:3.0:1789760741.295220:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720350611960 time 720399930373 state 0x0 ref 1 00000100:00100000:3.0:1789760741.295234:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508338048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760741.295329:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21016576, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:3.0:1789760741.295338:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5535c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.295356:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760741.295361:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.295369:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5535c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.295385:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5535c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.330721:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972451 is committed 00002000:00010000:3.0:1789760741.330866:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21016576 -> 21020672 00010000:00010000:3.0:1789760741.330883:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5535c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.330895:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5535c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.330904:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5535c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.330911:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760741.330916:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5535c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.330925:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5535c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.330949:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5535c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.330957:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5535c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21016576->21020671] (req 21016576->21020671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.331199:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508338048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35899us (36162us total) trans 4294972451 rc 0/0 00000100:00100000:3.0:1789760741.331213:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5163 00000100:00100000:3.0:1789760741.331224:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720350611960 time 720399930373 state 0x2 ref 0 00000100:00100000:3.0:1789760741.336957:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760741.337175:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508338176 00000100:00100000:0.0:1789760741.337186:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508338176 00000100:00100000:0.0:1789760741.337246:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720350611960 time 720399930373 state 0x0 ref 1 00000100:00100000:0.0:1789760741.337264:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720400190789 time 720399930373 state 0x1 ref 1 00000100:00100000:0.0:1789760741.337330:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720400190789 time 720442040595 state 0x0 ref 1 00000100:00100000:0.0:1789760741.337345:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508338176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760741.337395:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21020672, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:0.0:1789760741.337405:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55363 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.337656:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.337662:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.337672:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55363 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.337690:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55363 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.368947:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972452 is committed 00002000:00010000:2.0:1789760741.369111:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21020672 -> 21024768 00010000:00010000:2.0:1789760741.369128:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55363 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.369147:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55363 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.369156:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55363 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.369223:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760741.369229:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55363 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.369239:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55363 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.369264:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55363 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.369271:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55363 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21020672->21024767] (req 21020672->21024767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.369461:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508338176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32115us (32509us total) trans 4294972452 rc 0/0 00000100:00100000:2.0:1789760741.369472:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5164 00000100:00100000:2.0:1789760741.369482:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720400190789 time 720442040595 state 0x2 ref 0 00000100:00100000:1.0:1789760741.375777:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760741.375880:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508338304 00000100:00100000:2.0:1789760741.375892:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508338304 00000100:00100000:2.0:1789760741.375946:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720400190789 time 720442040595 state 0x0 ref 1 00000100:00100000:2.0:1789760741.375962:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720442301011 time 720442040595 state 0x1 ref 1 00000100:00100000:2.0:1789760741.376048:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720442301011 time 720480756619 state 0x0 ref 1 00000100:00100000:2.0:1789760741.376062:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508338304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760741.376125:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21024768, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:2.0:1789760741.376136:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5536a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.376157:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.376170:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.376178:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5536a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.376196:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5536a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760741.415177:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21024768 -> 21028864 00010000:00010000:2.0:1789760741.415203:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5536a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.415208:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972453 is committed 00010000:00010000:2.0:1789760741.415220:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5536a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.415235:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5536a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.415242:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760741.415247:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5536a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.415256:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5536a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.415278:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5536a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.415312:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5536a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21024768->21028863] (req 21024768->21028863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.415542:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508338304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39477us (39769us total) trans 4294972453 rc 0/0 00000100:00100000:2.0:1789760741.415551:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5165 00000100:00100000:2.0:1789760741.415562:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720442301011 time 720480756619 state 0x2 ref 0 00000100:00100000:0.0:1789760741.423614:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760741.423700:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508338432 00000100:00100000:3.0:1789760741.423710:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508338432 00000100:00100000:3.0:1789760741.423769:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720442301011 time 720480756619 state 0x0 ref 1 00000100:00100000:3.0:1789760741.423786:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720481017035 time 720480756619 state 0x1 ref 1 00000100:00100000:3.0:1789760741.423827:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720481017035 time 720528536561 state 0x0 ref 1 00000100:00100000:3.0:1789760741.423840:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508338432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760741.423892:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21028864, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:3.0:1789760741.423902:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55371 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.423918:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760741.423923:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.423930:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55371 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.423951:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55371 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.447761:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972454 is committed 00002000:00010000:3.0:1789760741.447917:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21028864 -> 21032960 00010000:00010000:3.0:1789760741.447934:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55371 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.447946:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55371 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.447954:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55371 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.447962:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760741.447967:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55371 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.447976:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55371 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.447999:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55371 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.448007:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55371 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21028864->21032959] (req 21028864->21032959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.448209:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508338432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24364us (24599us total) trans 4294972454 rc 0/0 00000100:00100000:3.0:1789760741.448220:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5166 00000100:00100000:3.0:1789760741.448230:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720481017035 time 720528536561 state 0x2 ref 0 00000100:00100000:3.0:1789760741.452971:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760741.453065:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508338560 00000100:00100000:2.0:1789760741.453086:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508338560 00000100:00100000:2.0:1789760741.453144:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720481017035 time 720528536561 state 0x0 ref 1 00000100:00100000:2.0:1789760741.453161:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720528796977 time 720528536561 state 0x1 ref 1 00000100:00100000:2.0:1789760741.453265:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720528796977 time 720557973833 state 0x0 ref 1 00000100:00100000:2.0:1789760741.453278:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508338560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760741.453511:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21032960, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:2.0:1789760741.453521:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55378 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.453537:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.453542:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.453549:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55378 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.453564:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55378 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.485121:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972455 is committed 00002000:00010000:0.0:1789760741.486734:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21032960 -> 21037056 00010000:00010000:0.0:1789760741.486753:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55378 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.486765:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55378 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.486774:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55378 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.486780:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760741.486785:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55378 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.486795:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55378 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.486877:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55378 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.486888:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55378 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21032960->21037055] (req 21032960->21037055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.487058:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508338560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33710us (34027us total) trans 4294972455 rc 0/0 00000100:00100000:0.0:1789760741.487069:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5167 00000100:00100000:0.0:1789760741.487091:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720528796977 time 720557973833 state 0x2 ref 0 00000100:00100000:1.0:1789760741.497016:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760741.497069:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508338688 00000100:00100000:2.0:1789760741.497090:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508338688 00000100:00100000:2.0:1789760741.497157:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720528796977 time 720557973833 state 0x0 ref 1 00000100:00100000:2.0:1789760741.497591:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720558234249 time 720557973833 state 0x1 ref 1 00000100:00100000:0.0:1789760741.497698:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720558234249 time 720602405379 state 0x0 ref 1 00000100:00100000:0.0:1789760741.497719:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508338688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760741.497769:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21037056, mtime: 0x6aad94e5, atime: 0x0, ctime: 0x6aad94e5, blocks: 0xa807 00010000:00010000:0.0:1789760741.497779:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5537f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.497796:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.497801:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.497808:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5537f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.497822:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5537f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.541966:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972456 is committed 00002000:00010000:0.0:1789760741.542091:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21037056 -> 21041152 00002000:00010000:0.0:1789760741.542097:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760741 -> 1789760742 00002000:00010000:0.0:1789760741.542100:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760741 -> 1789760742 00010000:00010000:0.0:1789760741.542123:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5537f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.542136:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5537f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.542145:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5537f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.542152:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760741.542158:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5537f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.542167:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5537f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.542191:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5537f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.542201:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5537f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21037056->21041151] (req 21037056->21041151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.542331:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508338688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44611us (45316us total) trans 4294972456 rc 0/0 00000100:00100000:0.0:1789760741.542341:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5168 00000100:00100000:0.0:1789760741.542351:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720558234249 time 720602405379 state 0x2 ref 0 00000100:00100000:1.0:1789760741.548137:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760741.548197:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508338816 00000100:00100000:3.0:1789760741.548207:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508338816 00000100:00100000:3.0:1789760741.548267:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720558234249 time 720602405379 state 0x0 ref 1 00000100:00100000:3.0:1789760741.548315:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720602665795 time 720602405379 state 0x1 ref 1 00000100:00100000:3.0:1789760741.548339:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720602665795 time 720653049941 state 0x0 ref 1 00000100:00100000:3.0:1789760741.548353:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508338816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760741.548409:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21041152, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:3.0:1789760741.548420:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55386 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.548498:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760741.548567:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.548578:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55386 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.548600:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55386 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.578502:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972457 is committed 00002000:00010000:3.0:1789760741.578563:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21041152 -> 21045248 00010000:00010000:3.0:1789760741.578579:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55386 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.578593:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55386 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.578603:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55386 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.578612:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760741.578617:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55386 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.578627:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55386 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.578650:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55386 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.578658:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55386 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21041152->21045247] (req 21041152->21045247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760741.578753:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508338816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30396us (30619us total) trans 4294972457 rc 0/0 00000100:00100000:3.0:1789760741.578765:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5169 00000100:00100000:3.0:1789760741.578776:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720602665795 time 720653049941 state 0x2 ref 0 00000100:00100000:3.0:1789760741.582476:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760741.582580:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508338944 00000100:00100000:0.0:1789760741.582591:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508338944 00000100:00100000:0.0:1789760741.582647:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720602665795 time 720653049941 state 0x0 ref 1 00000100:00100000:0.0:1789760741.582668:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720653310357 time 720653049941 state 0x1 ref 1 00000100:00100000:0.0:1789760741.582697:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720653310357 time 720687406817 state 0x0 ref 1 00000100:00100000:0.0:1789760741.582710:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508338944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760741.582773:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21045248, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:0.0:1789760741.582784:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5538d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.582800:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.582805:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.582812:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5538d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.582827:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5538d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.606396:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972458 is committed 00002000:00010000:1.0:1789760741.608275:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21045248 -> 21049344 00010000:00010000:1.0:1789760741.608308:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5538d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.608322:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5538d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.608331:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5538d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.608338:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760741.608342:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5538d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.608352:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5538d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.608374:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5538d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.608382:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5538d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21045248->21049343] (req 21045248->21049343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760741.608745:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508338944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26032us (26461us total) trans 4294972458 rc 0/0 00000100:00100000:1.0:1789760741.608755:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5170 00000100:00100000:1.0:1789760741.608766:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720653310357 time 720687406817 state 0x2 ref 0 00000100:00100000:0.0:1789760741.612595:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760741.612691:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508339072 00000100:00100000:2.0:1789760741.612703:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508339072 00000100:00100000:2.0:1789760741.612763:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720653310357 time 720687406817 state 0x0 ref 1 00000100:00100000:2.0:1789760741.612779:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720687667233 time 720687406817 state 0x1 ref 1 00000100:00100000:2.0:1789760741.612845:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720687667233 time 720717555279 state 0x0 ref 1 00000100:00100000:2.0:1789760741.612859:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508339072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760741.612911:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21049344, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:2.0:1789760741.612920:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55394 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.612938:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.612943:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.612949:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55394 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.612963:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55394 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.640727:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972459 is committed 00002000:00010000:2.0:1789760741.640900:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21049344 -> 21053440 00010000:00010000:2.0:1789760741.640917:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55394 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.640929:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55394 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.640939:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55394 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.640946:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760741.640952:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55394 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.640962:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55394 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.640985:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55394 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.640993:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55394 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21049344->21053439] (req 21049344->21053439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760741.641123:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508339072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28262us (28533us total) trans 4294972459 rc 0/0 00000100:00100000:2.0:1789760741.641135:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5171 00000100:00100000:2.0:1789760741.641145:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720687667233 time 720717555279 state 0x2 ref 0 00000100:00100000:3.0:1789760741.645017:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760741.645143:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508339200 00000100:00100000:0.0:1789760741.645153:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508339200 00000100:00100000:0.0:1789760741.645210:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720687667233 time 720717555279 state 0x0 ref 1 00000100:00100000:0.0:1789760741.645231:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720717815695 time 720717555279 state 0x1 ref 1 00000100:00100000:0.0:1789760741.645324:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720717815695 time 720750033293 state 0x0 ref 1 00000100:00100000:0.0:1789760741.645338:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508339200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760741.645394:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21053440, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:0.0:1789760741.645404:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5539b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.645421:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.645425:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.645432:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5539b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.645447:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5539b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.669531:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972460 is committed 00002000:00010000:0.0:1789760741.669647:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21053440 -> 21057536 00010000:00010000:0.0:1789760741.669663:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5539b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.669676:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5539b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.669685:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5539b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.669696:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760741.669702:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5539b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.669711:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5539b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.669735:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5539b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.669744:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5539b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21053440->21057535] (req 21053440->21057535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.669876:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508339200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24535us (24863us total) trans 4294972460 rc 0/0 00000100:00100000:0.0:1789760741.669886:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5172 00000100:00100000:0.0:1789760741.669897:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720717815695 time 720750033293 state 0x2 ref 0 00000100:00100000:2.0:1789760741.673200:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760741.673310:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508339328 00000100:00100000:1.0:1789760741.673321:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508339328 00000100:00100000:1.0:1789760741.673380:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720717815695 time 720750033293 state 0x0 ref 1 00000100:00100000:1.0:1789760741.673396:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720750293709 time 720750033293 state 0x1 ref 1 00000100:00100000:1.0:1789760741.673426:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720750293709 time 720778136024 state 0x0 ref 1 00000100:00100000:1.0:1789760741.673438:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508339328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760741.673487:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21057536, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:1.0:1789760741.673498:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe553a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.673515:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760741.673519:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760741.673526:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe553a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760741.673541:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe553a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760741.721092:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972461 is committed 00002000:00010000:0.0:1789760741.721184:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21057536 -> 21061632 00010000:00010000:0.0:1789760741.721200:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe553a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.721214:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe553a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.721229:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe553a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.721236:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760741.721242:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe553a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.721252:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe553a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.721273:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe553a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.721281:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe553a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21057536->21061631] (req 21057536->21061631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.721727:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508339328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48284us (48531us total) trans 4294972461 rc 0/0 00000100:00100000:0.0:1789760741.721737:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5173 00000100:00100000:0.0:1789760741.721748:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720750293709 time 720778136024 state 0x2 ref 0 00000100:00100000:1.0:1789760741.724875:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760741.725057:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508339456 00000100:00100000:3.0:1789760741.725068:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508339456 00000100:00100000:3.0:1789760741.725142:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720750293709 time 720778136024 state 0x0 ref 1 00000100:00100000:3.0:1789760741.725158:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720778396440 time 720778136024 state 0x1 ref 1 00000100:00100000:3.0:1789760741.725187:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720778396440 time 720829897961 state 0x0 ref 1 00000100:00100000:3.0:1789760741.725199:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508339456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760741.725250:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21061632, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:3.0:1789760741.725259:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe553a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.725275:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760741.725280:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760741.725324:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe553a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760741.725341:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe553a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.757128:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972462 is committed 00002000:00010000:0.0:1789760741.757226:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21061632 -> 21065728 00010000:00010000:0.0:1789760741.757246:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe553a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.757265:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe553a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.757274:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe553a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.757282:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760741.757316:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe553a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.757327:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe553a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.757352:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe553a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.757360:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe553a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21061632->21065727] (req 21061632->21065727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.757796:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508339456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32592us (32926us total) trans 4294972462 rc 0/0 00000100:00100000:0.0:1789760741.757808:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5174 00000100:00100000:0.0:1789760741.757820:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720778396440 time 720829897961 state 0x2 ref 0 00000100:00100000:3.0:1789760741.768821:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760741.768991:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508339584 00000100:00100000:0.0:1789760741.769002:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508339584 00000100:00100000:2.0:1789760741.769142:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720778396440 time 720829897961 state 0x0 ref 1 00000100:00100000:2.0:1789760741.769160:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720830158377 time 720829897961 state 0x1 ref 1 00000100:00100000:2.0:1789760741.769194:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720830158377 time 720873904232 state 0x0 ref 1 00000100:00100000:2.0:1789760741.769207:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508339584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760741.769261:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21065728, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:2.0:1789760741.769270:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe553b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.769967:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760741.769974:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.769983:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe553b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.770003:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe553b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.803098:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972463 is committed 00002000:00010000:0.0:1789760741.803196:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21065728 -> 21069824 00010000:00010000:0.0:1789760741.803213:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe553b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.803226:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe553b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.803235:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe553b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.803243:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760741.803248:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe553b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.803258:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe553b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.803282:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe553b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.803307:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe553b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21065728->21069823] (req 21065728->21069823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.803444:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508339584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34228us (34621us total) trans 4294972463 rc 0/0 00000100:00100000:0.0:1789760741.803455:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5175 00000100:00100000:0.0:1789760741.803467:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720830158377 time 720873904232 state 0x2 ref 0 00000100:00100000:1.0:1789760741.818230:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760741.818330:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508339712 00000100:00100000:0.0:1789760741.818341:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508339712 00000100:00100000:0.0:1789760741.818401:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720830158377 time 720873904232 state 0x0 ref 1 00000100:00100000:0.0:1789760741.818429:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720874164648 time 720873904232 state 0x1 ref 1 00000100:00100000:0.0:1789760741.818465:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720874164648 time 720923175073 state 0x0 ref 1 00000100:00100000:0.0:1789760741.818479:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508339712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760741.818529:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21069824, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:0.0:1789760741.818539:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.818557:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.818562:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.818569:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.818759:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.843771:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972464 is committed 00002000:00010000:0.0:1789760741.843810:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21069824 -> 21073920 00010000:00010000:0.0:1789760741.843827:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.843844:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.843854:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.843862:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760741.843867:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.843877:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.843900:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.843909:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21069824->21073919] (req 21069824->21073919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.844004:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508339712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25521us (25778us total) trans 4294972464 rc 0/0 00000100:00100000:0.0:1789760741.844015:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5176 00000100:00100000:0.0:1789760741.844024:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720874164648 time 720923175073 state 0x2 ref 0 00000100:00100000:1.0:1789760741.851344:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760741.854897:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508339840 00000100:00100000:0.0:1789760741.854912:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508339840 00000100:00100000:0.0:1789760741.854976:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720874164648 time 720923175073 state 0x0 ref 1 00000100:00100000:0.0:1789760741.854996:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720923435489 time 720923175073 state 0x1 ref 1 00000100:00100000:0.0:1789760741.855034:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720923435489 time 720959744127 state 0x0 ref 1 00000100:00100000:0.0:1789760741.855049:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508339840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760741.855124:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21073920, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:0.0:1789760741.855134:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.855151:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760741.855155:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.855163:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.855178:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.903261:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972465 is committed 00002000:00010000:0.0:1789760741.903566:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21073920 -> 21078016 00010000:00010000:0.0:1789760741.903583:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.903596:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.903611:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.903618:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760741.903623:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.903632:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.903657:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.903665:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21073920->21078015] (req 21073920->21078015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.903766:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508339840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48714us (52424us total) trans 4294972465 rc 0/0 00000100:00100000:0.0:1789760741.903777:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5177 00000100:00100000:0.0:1789760741.903787:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720923435489 time 720959744127 state 0x2 ref 0 00000100:00100000:3.0:1789760741.911164:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760741.911605:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508339968 00000100:00100000:0.0:1789760741.911615:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508339968 00000100:00100000:2.0:1789760741.911731:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720923435489 time 720959744127 state 0x0 ref 1 00000100:00100000:2.0:1789760741.911749:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720960004543 time 720959744127 state 0x1 ref 1 00000100:00100000:2.0:1789760741.911820:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720960004543 time 721016529439 state 0x0 ref 1 00000100:00100000:2.0:1789760741.911833:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508339968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760741.911886:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21078016, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:2.0:1789760741.911897:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe553c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.911915:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760741.911920:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760741.911927:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe553c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760741.911942:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe553c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760741.960014:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972466 is committed 00002000:00010000:0.0:1789760741.960245:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21078016 -> 21082112 00010000:00010000:0.0:1789760741.960263:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe553c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.960277:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe553c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.960303:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe553c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.960311:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760741.960316:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe553c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.960326:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe553c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.960348:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe553c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.960356:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe553c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21078016->21082111] (req 21078016->21082111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.960572:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508339968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48735us (49414us total) trans 4294972466 rc 0/0 00000100:00100000:0.0:1789760741.961745:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5178 00000100:00100000:0.0:1789760741.961757:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720960004543 time 721016529439 state 0x2 ref 0 00000100:00100000:1.0:1789760741.966882:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760741.967508:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508340096 00000100:00100000:0.0:1789760741.967520:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508340096 00000100:00100000:0.0:1789760741.967591:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 720960004543 time 721016529439 state 0x0 ref 1 00000100:00100000:0.0:1789760741.967609:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721016789855 time 721016529439 state 0x1 ref 1 00000100:00100000:0.0:1789760741.967687:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721016789855 time 721072396453 state 0x0 ref 1 00000100:00100000:0.0:1789760741.967701:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508340096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760741.967755:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21082112, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:0.0:1789760741.967765:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.967781:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760741.967786:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.967793:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.967819:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760741.993588:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972467 is committed 00002000:00010000:0.0:1789760741.993687:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21082112 -> 21086208 00010000:00010000:0.0:1789760741.993703:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.993716:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.993729:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.993737:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760741.993742:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.993752:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.993772:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.993780:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21082112->21086207] (req 21082112->21086207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760741.993870:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508340096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26166us (26992us total) trans 4294972467 rc 0/0 00000100:00100000:0.0:1789760741.993880:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5179 00000100:00100000:0.0:1789760741.993891:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721016789855 time 721072396453 state 0x2 ref 0 00000100:00100000:1.0:1789760741.998670:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760741.998803:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508340224 00000100:00100000:0.0:1789760741.998813:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508340224 00000100:00100000:0.0:1789760741.998875:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721016789855 time 721072396453 state 0x0 ref 1 00000100:00100000:0.0:1789760741.998892:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721072656869 time 721072396453 state 0x1 ref 1 00000100:00100000:0.0:1789760741.999009:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721072656869 time 721103718655 state 0x0 ref 1 00000100:00100000:0.0:1789760741.999024:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508340224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760741.999088:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21086208, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:0.0:1789760741.999101:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe553d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.999119:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760741.999124:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760741.999131:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe553d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760741.999147:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe553d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.049838:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972468 is committed 00002000:00010000:1.0:1789760742.049949:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21086208 -> 21090304 00010000:00010000:1.0:1789760742.049967:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe553d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.049981:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe553d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.049990:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe553d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.049998:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760742.050003:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe553d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.050012:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe553d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.050036:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe553d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.050044:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe553d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21086208->21090303] (req 21086208->21090303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.050188:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508340224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51161us (51522us total) trans 4294972468 rc 0/0 00000100:00100000:1.0:1789760742.050200:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5180 00000100:00100000:1.0:1789760742.050210:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721072656869 time 721103718655 state 0x2 ref 0 00000100:00100000:2.0:1789760742.057393:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760742.057476:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508340352 00000100:00100000:0.0:1789760742.057487:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508340352 00000100:00100000:0.0:1789760742.057554:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721072656869 time 721103718655 state 0x0 ref 1 00000100:00100000:0.0:1789760742.057575:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721103979071 time 721103718655 state 0x1 ref 1 00000100:00100000:0.0:1789760742.057604:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721103979071 time 721162314348 state 0x0 ref 1 00000100:00100000:0.0:1789760742.057616:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508340352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760742.057680:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21090304, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:0.0:1789760742.057694:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.057711:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760742.057716:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.057723:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.057738:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.094275:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972469 is committed 00002000:00010000:2.0:1789760742.094646:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21090304 -> 21094400 00010000:00010000:2.0:1789760742.094664:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.094678:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.094687:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.094694:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760742.094699:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.094708:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.094730:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.094738:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe553da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21090304->21094399] (req 21090304->21094399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.095056:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508340352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37436us (37667us total) trans 4294972469 rc 0/0 00000100:00100000:2.0:1789760742.095066:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5181 00000100:00100000:2.0:1789760742.095087:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721103979071 time 721162314348 state 0x2 ref 0 00000100:00100000:1.0:1789760742.099435:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760742.099699:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508340480 00000100:00100000:0.0:1789760742.099710:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508340480 00000100:00100000:0.0:1789760742.099767:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721103979071 time 721162314348 state 0x0 ref 1 00000100:00100000:0.0:1789760742.099784:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721162574764 time 721162314348 state 0x1 ref 1 00000100:00100000:0.0:1789760742.099818:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721162574764 time 721204528023 state 0x0 ref 1 00000100:00100000:0.0:1789760742.099830:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508340480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760742.099882:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21094400, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:0.0:1789760742.099892:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe553e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.099909:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760742.099914:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.099921:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe553e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.099936:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe553e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.130717:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972470 is committed 00002000:00010000:2.0:1789760742.130820:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21094400 -> 21098496 00010000:00010000:2.0:1789760742.130838:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe553e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.130851:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe553e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.130860:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe553e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.130867:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760742.130871:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe553e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.130881:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe553e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.130908:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe553e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.130917:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe553e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21094400->21098495] (req 21094400->21098495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.131135:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508340480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31300us (31715us total) trans 4294972470 rc 0/0 00000100:00100000:2.0:1789760742.131146:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5182 00000100:00100000:2.0:1789760742.131157:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721162574764 time 721204528023 state 0x2 ref 0 00000100:00100000:1.0:1789760742.138581:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760742.138675:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508340608 00000100:00100000:0.0:1789760742.138686:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508340608 00000100:00100000:0.0:1789760742.138747:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721162574764 time 721204528023 state 0x0 ref 1 00000100:00100000:0.0:1789760742.138764:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721204788439 time 721204528023 state 0x1 ref 1 00000100:00100000:0.0:1789760742.138830:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721204788439 time 721243539914 state 0x0 ref 1 00000100:00100000:0.0:1789760742.138844:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508340608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760742.138897:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21098496, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:0.0:1789760742.138907:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe553e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.138924:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760742.138929:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.138936:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe553e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.138952:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe553e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.142852:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760742.171947:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972471 is committed 00002000:00010000:1.0:1789760742.172089:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21098496 -> 21102592 00010000:00010000:1.0:1789760742.172108:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe553e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.172121:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe553e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.172131:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe553e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.172139:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760742.172145:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe553e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.172158:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe553e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.172181:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe553e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.172190:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe553e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21098496->21102591] (req 21098496->21102591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.172486:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508340608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33639us (33909us total) trans 4294972471 rc 0/0 00000100:00100000:1.0:1789760742.172497:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5183 00000100:00100000:1.0:1789760742.172507:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721204788439 time 721243539914 state 0x2 ref 0 00000100:00100000:3.0:1789760742.176560:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760742.176677:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508340736 00000100:00100000:2.0:1789760742.176688:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508340736 00000100:00100000:2.0:1789760742.176745:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721204788439 time 721243539914 state 0x0 ref 1 00000100:00100000:2.0:1789760742.176762:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721243800330 time 721243539914 state 0x1 ref 1 00000100:00100000:2.0:1789760742.176789:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721243800330 time 721281499259 state 0x0 ref 1 00000100:00100000:2.0:1789760742.176801:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508340736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760742.176855:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21102592, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:2.0:1789760742.176866:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe553ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.176884:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.176889:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.176895:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe553ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.176910:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe553ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760742.213886:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972472 is committed 00002000:00010000:1.0:1789760742.214157:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21102592 -> 21106688 00010000:00010000:1.0:1789760742.214174:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe553ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.214187:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe553ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.214196:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe553ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.214202:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760742.214209:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe553ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.214218:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe553ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.214242:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe553ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760742.214250:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe553ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21102592->21106687] (req 21102592->21106687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760742.214387:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508340736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37583us (37831us total) trans 4294972472 rc 0/0 00000100:00100000:1.0:1789760742.214398:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5184 00000100:00100000:1.0:1789760742.214408:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721243800330 time 721281499259 state 0x2 ref 0 00000100:00100000:3.0:1789760742.218746:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760742.218945:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508340864 00000100:00100000:0.0:1789760742.218955:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508340864 00000100:00100000:0.0:1789760742.219013:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721243800330 time 721281499259 state 0x0 ref 1 00000100:00100000:0.0:1789760742.219029:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721281759675 time 721281499259 state 0x1 ref 1 00000100:00100000:0.0:1789760742.219059:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721281759675 time 721323769605 state 0x0 ref 1 00000100:00100000:0.0:1789760742.219091:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508340864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760742.219140:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21106688, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:0.0:1789760742.219149:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe553f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.219167:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760742.219172:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.219179:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe553f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.219194:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe553f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.242883:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972473 is committed 00002000:00010000:0.0:1789760742.243134:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21106688 -> 21110784 00010000:00010000:0.0:1789760742.243152:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe553f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.243164:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe553f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.243178:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe553f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.243184:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760742.243189:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe553f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.243198:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe553f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.243220:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe553f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.243228:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe553f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21106688->21110783] (req 21106688->21110783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.243356:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508340864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24264us (24614us total) trans 4294972473 rc 0/0 00000100:00100000:0.0:1789760742.243366:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5185 00000100:00100000:0.0:1789760742.243388:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721281759675 time 721323769605 state 0x2 ref 0 00000100:00100000:1.0:1789760742.249717:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760742.250212:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508340992 00000100:00100000:0.0:1789760742.250223:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508340992 00000100:00100000:0.0:1789760742.250280:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721281759675 time 721323769605 state 0x0 ref 1 00000100:00100000:0.0:1789760742.250332:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721324030021 time 721323769605 state 0x1 ref 1 00000100:00100000:3.0:1789760742.250434:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721324030021 time 721355140682 state 0x0 ref 1 00000100:00100000:3.0:1789760742.250450:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508340992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760742.250507:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21110784, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:3.0:1789760742.250517:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe553fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.250534:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.250538:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.250546:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe553fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.250562:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe553fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.270999:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b6c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534070656:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760742.271085:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760742.271897:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534070656 00000100:00100000:0.0:1789760742.271908:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534070656 00000100:00100000:0.0:1789760742.272002:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:0.0:1789760742.272014:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534070656:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760742.272054:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789760742.272188:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534070656:12345-0@lo:41:osp-pre-0-1.0 Request processed in 176us (1103us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760742.272196:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00100000:1.0:1789760742.272279:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b6c0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534070656:0@lo:41:osp-pre-0-1.0 00002000:00010000:0.0:1789760742.291185:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21110784 -> 21114880 00010000:00010000:0.0:1789760742.291206:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe553fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.291219:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe553fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.291227:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe553fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.291235:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760742.291240:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe553fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.291249:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe553fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.291271:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe553fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.291280:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe553fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21110784->21114879] (req 21110784->21114879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.291401:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508340992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40950us (41690us total) trans 4294972474 rc 0/0 00000100:00100000:0.0:1789760742.291412:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5186 00000100:00100000:0.0:1789760742.291432:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721324030021 time 721355140682 state 0x2 ref 0 00000001:00080000:2.0:1789760742.291564:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972474 is committed 00000100:00100000:1.0:1789760742.295386:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760742.297131:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508341120 00000100:00100000:3.0:1789760742.297145:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508341120 00000100:00100000:3.0:1789760742.297214:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721324030021 time 721355140682 state 0x0 ref 1 00000100:00100000:3.0:1789760742.297230:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721355401098 time 721355140682 state 0x1 ref 1 00000100:00100000:3.0:1789760742.297320:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721355401098 time 721402029265 state 0x0 ref 1 00000100:00100000:3.0:1789760742.297335:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508341120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760742.297388:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21114880, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:3.0:1789760742.297398:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55404 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.297427:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.297433:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.297440:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55404 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.297458:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55404 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.321348:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972475 is committed 00002000:00010000:3.0:1789760742.321506:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21114880 -> 21118976 00010000:00010000:3.0:1789760742.321522:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55404 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.321536:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55404 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.321545:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55404 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.321553:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760742.321559:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55404 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.321569:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55404 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.321592:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55404 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.321601:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55404 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21114880->21118975] (req 21114880->21118975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.321699:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508341120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24360us (26316us total) trans 4294972475 rc 0/0 00000100:00100000:3.0:1789760742.321709:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5187 00000100:00100000:3.0:1789760742.321719:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721355401098 time 721402029265 state 0x2 ref 0 00000100:00100000:1.0:1789760742.325761:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760742.325872:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508341248 00000100:00100000:0.0:1789760742.325884:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508341248 00000100:00100000:0.0:1789760742.325943:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721355401098 time 721402029265 state 0x0 ref 1 00000100:00100000:0.0:1789760742.325961:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721402289681 time 721402029265 state 0x1 ref 1 00000100:00100000:0.0:1789760742.326088:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721402289681 time 721430796899 state 0x0 ref 1 00000100:00100000:0.0:1789760742.326101:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508341248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760742.326155:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21118976, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:0.0:1789760742.326165:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5540b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.326183:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760742.326188:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.326195:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5540b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.326210:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5540b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.351467:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972476 is committed 00002000:00010000:2.0:1789760742.351910:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21118976 -> 21123072 00010000:00010000:2.0:1789760742.351927:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5540b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.351941:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5540b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.351950:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5540b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.351957:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760742.351963:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5540b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.351973:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5540b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.352004:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5540b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.352015:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5540b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21118976->21123071] (req 21118976->21123071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.352134:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508341248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26029us (26379us total) trans 4294972476 rc 0/0 00000100:00100000:2.0:1789760742.352144:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5188 00000100:00100000:2.0:1789760742.352155:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721402289681 time 721430796899 state 0x2 ref 0 00000100:00100000:0.0:1789760742.355787:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760742.355898:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508341376 00000100:00100000:3.0:1789760742.355909:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508341376 00000100:00100000:3.0:1789760742.355963:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721402289681 time 721430796899 state 0x0 ref 1 00000100:00100000:3.0:1789760742.355977:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721431057315 time 721430796899 state 0x1 ref 1 00000100:00100000:3.0:1789760742.356003:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721431057315 time 721460713199 state 0x0 ref 1 00000100:00100000:3.0:1789760742.356015:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508341376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760742.356075:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21123072, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:3.0:1789760742.356086:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55412 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.356103:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.356107:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.356114:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55412 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.356135:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55412 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.389650:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972477 is committed 00002000:00010000:2.0:1789760742.390118:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21123072 -> 21127168 00010000:00010000:2.0:1789760742.390134:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55412 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.390147:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55412 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.390156:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55412 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.390163:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760742.390168:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55412 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.390178:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55412 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.390205:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55412 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.390214:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55412 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21123072->21127167] (req 21123072->21127167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.390347:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508341376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34327us (34565us total) trans 4294972477 rc 0/0 00000100:00100000:2.0:1789760742.390358:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5189 00000100:00100000:2.0:1789760742.390368:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721431057315 time 721460713199 state 0x2 ref 0 00000100:00100000:1.0:1789760742.398462:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760742.398562:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508341504 00000100:00100000:3.0:1789760742.398572:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508341504 00000100:00100000:3.0:1789760742.398633:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721431057315 time 721460713199 state 0x0 ref 1 00000100:00100000:3.0:1789760742.398650:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721460973615 time 721460713199 state 0x1 ref 1 00000100:00100000:3.0:1789760742.398687:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721460973615 time 721503397032 state 0x0 ref 1 00000100:00100000:3.0:1789760742.398700:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508341504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760742.398754:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21127168, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:3.0:1789760742.398764:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55419 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.398783:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.398787:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.398794:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55419 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.398810:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55419 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.442237:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972478 is committed 00002000:00010000:2.0:1789760742.442597:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21127168 -> 21131264 00010000:00010000:2.0:1789760742.442614:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55419 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.442626:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55419 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.442635:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55419 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.442643:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760742.442648:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55419 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.442663:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55419 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.442687:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55419 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.442696:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55419 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21127168->21131263] (req 21127168->21131263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.442829:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508341504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44125us (44371us total) trans 4294972478 rc 0/0 00000100:00100000:2.0:1789760742.442841:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5190 00000100:00100000:2.0:1789760742.442851:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721460973615 time 721503397032 state 0x2 ref 0 00000100:00100000:0.0:1789760742.446855:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760742.446948:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508341632 00000100:00100000:2.0:1789760742.446958:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508341632 00000100:00100000:2.0:1789760742.447098:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721460973615 time 721503397032 state 0x0 ref 1 00000100:00100000:2.0:1789760742.447115:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721503657448 time 721503397032 state 0x1 ref 1 00000100:00100000:2.0:1789760742.447188:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721503657448 time 721551897692 state 0x0 ref 1 00000100:00100000:2.0:1789760742.447203:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508341632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760742.447256:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21131264, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:2.0:1789760742.447266:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55420 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.447318:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.447323:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.447331:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55420 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.447346:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55420 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.493514:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972479 is committed 00002000:00010000:2.0:1789760742.494138:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21131264 -> 21135360 00010000:00010000:2.0:1789760742.494156:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55420 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.494563:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55420 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.494577:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55420 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.494584:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760742.494590:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55420 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.494599:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55420 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.494621:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55420 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.494630:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55420 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21131264->21135359] (req 21131264->21135359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.494738:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508341632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47532us (47887us total) trans 4294972479 rc 0/0 00000100:00100000:2.0:1789760742.494749:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5191 00000100:00100000:2.0:1789760742.494760:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721503657448 time 721551897692 state 0x2 ref 0 00000100:00100000:0.0:1789760742.506488:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760742.506632:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508341760 00000100:00100000:2.0:1789760742.506643:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508341760 00000100:00100000:2.0:1789760742.506705:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721503657448 time 721551897692 state 0x0 ref 1 00000100:00100000:2.0:1789760742.506722:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721552158108 time 721551897692 state 0x1 ref 1 00000100:00100000:2.0:1789760742.506753:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721552158108 time 721611463859 state 0x0 ref 1 00000100:00100000:2.0:1789760742.506767:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508341760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760742.506818:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21135360, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0xa807 00010000:00010000:2.0:1789760742.506829:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55427 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.506847:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.506852:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.506858:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55427 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.506878:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55427 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.532999:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972480 is committed 00002000:00010000:2.0:1789760742.533112:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21135360 -> 21139456 00002000:00010000:2.0:1789760742.533116:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760742 -> 1789760743 00002000:00010000:2.0:1789760742.533118:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760742 -> 1789760743 00010000:00010000:2.0:1789760742.533132:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55427 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.533143:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55427 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.533153:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55427 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.533160:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760742.533164:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55427 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.533173:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55427 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.533195:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55427 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.533206:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55427 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21135360->21139455] (req 21135360->21139455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.533323:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508341760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26554us (26841us total) trans 4294972480 rc 0/0 00000100:00100000:2.0:1789760742.533332:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5192 00000100:00100000:2.0:1789760742.533342:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721552158108 time 721611463859 state 0x2 ref 0 00000100:00100000:1.0:1789760742.536754:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760742.536842:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508341888 00000100:00100000:2.0:1789760742.536854:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508341888 00000100:00100000:2.0:1789760742.536914:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721552158108 time 721611463859 state 0x0 ref 1 00000100:00100000:2.0:1789760742.536930:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721611724275 time 721611463859 state 0x1 ref 1 00000100:00100000:2.0:1789760742.537001:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721611724275 time 721641710951 state 0x0 ref 1 00000100:00100000:2.0:1789760742.537015:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508341888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760742.537075:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21139456, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:2.0:1789760742.537085:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5542e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.537103:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.537108:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.537115:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5542e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.537136:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5542e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.560661:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972481 is committed 00002000:00010000:2.0:1789760742.560873:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21139456 -> 21143552 00010000:00010000:2.0:1789760742.560888:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5542e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.560901:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5542e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.560909:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5542e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.560917:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760742.560922:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5542e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.560931:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5542e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.560953:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5542e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.560961:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5542e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21139456->21143551] (req 21139456->21143551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.561169:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508341888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24151us (24419us total) trans 4294972481 rc 0/0 00000100:00100000:2.0:1789760742.561180:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5193 00000100:00100000:2.0:1789760742.561191:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721611724275 time 721641710951 state 0x2 ref 0 00000100:00100000:1.0:1789760742.565851:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760742.565974:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508342016 00000100:00100000:2.0:1789760742.565996:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508342016 00000100:00100000:2.0:1789760742.566060:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721611724275 time 721641710951 state 0x0 ref 1 00000100:00100000:2.0:1789760742.566086:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721641971367 time 721641710951 state 0x1 ref 1 00000100:00100000:2.0:1789760742.566153:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721641971367 time 721670859972 state 0x0 ref 1 00000100:00100000:2.0:1789760742.566167:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508342016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760742.566217:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21143552, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:2.0:1789760742.566227:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55435 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.566244:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.566249:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.566255:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55435 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.566271:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55435 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.590879:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972482 is committed 00002000:00010000:3.0:1789760742.591126:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21143552 -> 21147648 00010000:00010000:3.0:1789760742.591142:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55435 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.591155:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55435 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.591164:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55435 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.591177:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760742.591182:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55435 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.591191:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55435 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.591213:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55435 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.591221:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55435 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21143552->21147647] (req 21143552->21147647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.591422:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508342016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25251us (25579us total) trans 4294972482 rc 0/0 00000100:00100000:3.0:1789760742.591433:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5194 00000100:00100000:3.0:1789760742.591444:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721641971367 time 721670859972 state 0x2 ref 0 00000100:00100000:0.0:1789760742.594947:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760742.595217:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508342144 00000100:00100000:2.0:1789760742.595228:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508342144 00000100:00100000:2.0:1789760742.595314:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721641971367 time 721670859972 state 0x0 ref 1 00000100:00100000:2.0:1789760742.595333:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721671120388 time 721670859972 state 0x1 ref 1 00000100:00100000:2.0:1789760742.595363:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721671120388 time 721700073117 state 0x0 ref 1 00000100:00100000:2.0:1789760742.595404:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508342144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760742.595557:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21147648, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:2.0:1789760742.595568:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5543c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.595586:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.595590:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.595598:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5543c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.595611:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5543c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760742.625473:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21147648 -> 21151744 00010000:00010000:3.0:1789760742.625496:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5543c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.625511:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5543c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.625526:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5543c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.625535:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760742.625541:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5543c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.625550:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5543c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.625573:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5543c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.625582:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5543c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21147648->21151743] (req 21147648->21151743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760742.625681:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508342144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30277us (30735us total) trans 4294972483 rc 0/0 00000100:00100000:3.0:1789760742.625691:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5195 00000100:00100000:3.0:1789760742.625701:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721671120388 time 721700073117 state 0x2 ref 0 00000001:00080000:0.0:1789760742.626186:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972483 is committed 00000100:00100000:3.0:1789760742.630840:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760742.631140:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508342272 00000100:00100000:2.0:1789760742.631152:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508342272 00000100:00100000:2.0:1789760742.631211:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721671120388 time 721700073117 state 0x0 ref 1 00000100:00100000:2.0:1789760742.631229:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721700333533 time 721700073117 state 0x1 ref 1 00000100:00100000:2.0:1789760742.631375:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721700333533 time 721736084929 state 0x0 ref 1 00000100:00100000:2.0:1789760742.631389:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508342272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760742.631443:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21151744, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:2.0:1789760742.631454:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55443 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.631473:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.631479:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.631486:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55443 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.631502:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55443 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.660849:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972484 is committed 00002000:00010000:0.0:1789760742.660923:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21151744 -> 21155840 00010000:00010000:0.0:1789760742.660939:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55443 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.660951:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55443 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.660959:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55443 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.660967:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760742.660972:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55443 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.660982:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55443 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.661011:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55443 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.661018:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55443 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21151744->21155839] (req 21151744->21155839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.661120:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508342272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29727us (30283us total) trans 4294972484 rc 0/0 00000100:00100000:0.0:1789760742.661130:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5196 00000100:00100000:0.0:1789760742.661141:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721700333533 time 721736084929 state 0x2 ref 0 00000100:00100000:1.0:1789760742.666458:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760742.666541:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508342400 00000100:00100000:3.0:1789760742.666552:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508342400 00000100:00100000:3.0:1789760742.666610:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721700333533 time 721736084929 state 0x0 ref 1 00000100:00100000:3.0:1789760742.666629:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721736345345 time 721736084929 state 0x1 ref 1 00000100:00100000:3.0:1789760742.666669:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721736345345 time 721771371879 state 0x0 ref 1 00000100:00100000:3.0:1789760742.666683:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508342400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760742.666734:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21155840, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:3.0:1789760742.666743:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5544a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.666761:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.666767:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.666774:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5544a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.666792:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5544a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760742.703177:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972485 is committed 00002000:00010000:2.0:1789760742.703332:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21155840 -> 21159936 00010000:00010000:2.0:1789760742.703349:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5544a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.703362:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5544a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.703371:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5544a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.703379:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760742.703385:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5544a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.703395:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5544a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.703427:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5544a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.703436:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5544a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21155840->21159935] (req 21155840->21159935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.703531:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508342400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36844us (37077us total) trans 4294972485 rc 0/0 00000100:00100000:2.0:1789760742.703542:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5197 00000100:00100000:2.0:1789760742.703553:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721736345345 time 721771371879 state 0x2 ref 0 00000100:00100000:1.0:1789760742.706946:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760742.707102:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508342528 00000100:00100000:3.0:1789760742.707114:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508342528 00000100:00100000:3.0:1789760742.707169:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721736345345 time 721771371879 state 0x0 ref 1 00000100:00100000:3.0:1789760742.707184:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721771632295 time 721771371879 state 0x1 ref 1 00000100:00100000:3.0:1789760742.707244:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721771632295 time 721811953866 state 0x0 ref 1 00000100:00100000:3.0:1789760742.707266:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508342528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760742.707339:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21159936, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:3.0:1789760742.707348:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55451 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.707364:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.707369:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.707375:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55451 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.707389:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55451 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.748085:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972486 is committed 00002000:00010000:0.0:1789760742.748186:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21159936 -> 21164032 00010000:00010000:0.0:1789760742.748204:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55451 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.748217:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55451 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.748227:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55451 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.748234:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760742.748239:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55451 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.748253:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55451 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.748371:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55451 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.748378:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55451 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21159936->21164031] (req 21159936->21164031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.748616:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508342528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41355us (41675us total) trans 4294972486 rc 0/0 00000100:00100000:0.0:1789760742.748625:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5198 00000100:00100000:0.0:1789760742.748635:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721771632295 time 721811953866 state 0x2 ref 0 00000100:00100000:0.0:1789760742.755436:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760742.755534:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508342656 00000100:00100000:2.0:1789760742.755545:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508342656 00000100:00100000:2.0:1789760742.755607:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721771632295 time 721811953866 state 0x0 ref 1 00000100:00100000:2.0:1789760742.755624:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721812214282 time 721811953866 state 0x1 ref 1 00000100:00100000:2.0:1789760742.755656:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721812214282 time 721860365926 state 0x0 ref 1 00000100:00100000:2.0:1789760742.755669:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508342656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760742.755725:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21164032, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:2.0:1789760742.755736:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55458 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.755754:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.755759:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.755766:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55458 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.755782:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55458 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.783721:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972487 is committed 00002000:00010000:0.0:1789760742.783827:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21164032 -> 21168128 00010000:00010000:0.0:1789760742.783845:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55458 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.783858:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55458 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.783880:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55458 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.783893:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760742.783899:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55458 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.783909:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55458 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.783934:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55458 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.783942:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55458 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21164032->21168127] (req 21164032->21168127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.784040:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508342656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28367us (28609us total) trans 4294972487 rc 0/0 00000100:00100000:0.0:1789760742.784051:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5199 00000100:00100000:0.0:1789760742.784071:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721812214282 time 721860365926 state 0x2 ref 0 00000100:00100000:1.0:1789760742.795366:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760742.795566:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508342784 00000100:00100000:2.0:1789760742.795587:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508342784 00000100:00100000:2.0:1789760742.795645:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721812214282 time 721860365926 state 0x0 ref 1 00000100:00100000:2.0:1789760742.795663:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721860626342 time 721860365926 state 0x1 ref 1 00000100:00100000:2.0:1789760742.795694:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721860626342 time 721900403522 state 0x0 ref 1 00000100:00100000:2.0:1789760742.795708:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508342784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760742.795759:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21168128, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:2.0:1789760742.795768:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5545f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.795784:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760742.795788:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.795794:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5545f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.795808:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5545f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760742.826666:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21168128 -> 21172224 00010000:00010000:2.0:1789760742.826688:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5545f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.826702:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5545f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.826712:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5545f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.826721:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760742.826726:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5545f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.826736:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5545f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.826760:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5545f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.826768:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe5545f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21168128->21172223] (req 21168128->21172223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760742.826871:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508342784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31160us (31509us total) trans 4294972488 rc 0/0 00000100:00100000:2.0:1789760742.826882:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5200 00000100:00100000:2.0:1789760742.826893:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721860626342 time 721900403522 state 0x2 ref 0 00000001:00080000:3.0:1789760742.827885:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972488 is committed 00000100:00100000:3.0:1789760742.830124:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760742.830198:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508342912 00000100:00100000:0.0:1789760742.830209:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508342912 00000100:00100000:0.0:1789760742.830266:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721860626342 time 721900403522 state 0x0 ref 1 00000100:00100000:0.0:1789760742.830282:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721900663938 time 721900403522 state 0x1 ref 1 00000100:00100000:0.0:1789760742.830345:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721900663938 time 721935054967 state 0x0 ref 1 00000100:00100000:0.0:1789760742.830360:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508342912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760742.830413:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21172224, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:0.0:1789760742.830424:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55466 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.830442:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760742.830447:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.830453:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55466 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.830469:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55466 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.868916:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972489 is committed 00002000:00010000:0.0:1789760742.869005:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21172224 -> 21176320 00010000:00010000:0.0:1789760742.869020:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55466 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.869033:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55466 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.869042:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55466 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.869050:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760742.869055:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55466 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.869075:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55466 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.869100:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55466 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.869109:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55466 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21172224->21176319] (req 21172224->21176319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.869203:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508342912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38840us (39083us total) trans 4294972489 rc 0/0 00000100:00100000:0.0:1789760742.869214:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5201 00000100:00100000:0.0:1789760742.869224:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721900663938 time 721935054967 state 0x2 ref 0 00000100:00100000:0.0:1789760742.876170:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760742.876270:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508343040 00000100:00100000:3.0:1789760742.876283:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508343040 00000100:00100000:3.0:1789760742.876391:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721900663938 time 721935054967 state 0x0 ref 1 00000100:00100000:3.0:1789760742.876408:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721935315383 time 721935054967 state 0x1 ref 1 00000100:00100000:3.0:1789760742.876450:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721935315383 time 721981160753 state 0x0 ref 1 00000100:00100000:3.0:1789760742.876464:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508343040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760742.876516:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21176320, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:3.0:1789760742.876526:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5546d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.876545:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760742.876549:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760742.876556:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5546d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760742.876570:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5546d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760742.912407:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972490 is committed 00002000:00010000:0.0:1789760742.912744:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21176320 -> 21180416 00010000:00010000:0.0:1789760742.912763:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5546d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.912777:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5546d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.912787:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5546d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.912873:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760742.912880:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5546d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.912891:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5546d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.912922:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5546d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.912930:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5546d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21176320->21180415] (req 21176320->21180415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.913089:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508343040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36622us (36924us total) trans 4294972490 rc 0/0 00000100:00100000:0.0:1789760742.913099:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5202 00000100:00100000:0.0:1789760742.913110:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721935315383 time 721981160753 state 0x2 ref 0 00000100:00100000:1.0:1789760742.920610:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760742.920711:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508343168 00000100:00100000:2.0:1789760742.920722:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508343168 00000100:00100000:2.0:1789760742.920786:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721935315383 time 721981160753 state 0x0 ref 1 00000100:00100000:2.0:1789760742.920815:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721981421169 time 721981160753 state 0x1 ref 1 00000100:00100000:2.0:1789760742.920844:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721981421169 time 722025554237 state 0x0 ref 1 00000100:00100000:2.0:1789760742.920857:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508343168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760742.920908:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21180416, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:2.0:1789760742.920918:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55474 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.920934:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760742.920938:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760742.920945:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55474 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760742.920961:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55474 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760742.949357:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21180416 -> 21184512 00010000:00010000:0.0:1789760742.949378:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55474 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.949391:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55474 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.949407:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55474 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.949426:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760742.949432:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55474 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.949441:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55474 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.949464:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55474 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.949472:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55474 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21180416->21184511] (req 21180416->21184511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760742.949575:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508343168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28715us (28971us total) trans 4294972491 rc 0/0 00000100:00100000:0.0:1789760742.949586:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5203 00000100:00100000:0.0:1789760742.949597:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721981421169 time 722025554237 state 0x2 ref 0 00000001:00080000:0.0:1789760742.950378:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972491 is committed 00000100:00100000:3.0:1789760742.953501:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760742.953553:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508343296 00000100:00100000:0.0:1789760742.953565:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508343296 00000100:00100000:0.0:1789760742.953631:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 721981421169 time 722025554237 state 0x0 ref 1 00000100:00100000:0.0:1789760742.953652:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722025814653 time 722025554237 state 0x1 ref 1 00000100:00100000:0.0:1789760742.953683:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722025814653 time 722058393423 state 0x0 ref 1 00000100:00100000:0.0:1789760742.953696:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508343296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760742.953749:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21184512, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:0.0:1789760742.953759:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5547b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.953777:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760742.953781:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.953789:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5547b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.953805:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5547b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760742.991726:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21184512 -> 21188608 00010000:00010000:0.0:1789760742.991746:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5547b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.991760:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5547b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.991769:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5547b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.991783:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760742.991789:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5547b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.991797:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5547b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.991821:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5547b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.991830:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5547b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21184512->21188607] (req 21184512->21188607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760742.992498:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972492 is committed 00000100:00100000:0.0:1789760742.992666:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508343296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38966us (39169us total) trans 4294972492 rc 0/0 00000100:00100000:0.0:1789760742.992678:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5204 00000100:00100000:0.0:1789760742.992689:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722025814653 time 722058393423 state 0x2 ref 0 00000100:00100000:1.0:1789760742.997631:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760742.997763:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508343424 00000100:00100000:0.0:1789760742.997775:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508343424 00000100:00100000:0.0:1789760742.997834:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722025814653 time 722058393423 state 0x0 ref 1 00000100:00100000:0.0:1789760742.997853:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722058653839 time 722058393423 state 0x1 ref 1 00000100:00100000:0.0:1789760742.997885:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722058653839 time 722102595503 state 0x0 ref 1 00000100:00100000:0.0:1789760742.997903:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508343424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760742.997952:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21188608, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:0.0:1789760742.997963:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55482 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.997981:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760742.997986:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760742.997993:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55482 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760742.998010:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55482 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.033588:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972493 is committed 00002000:00010000:1.0:1789760743.036743:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21188608 -> 21192704 00010000:00010000:1.0:1789760743.036763:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55482 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.036776:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55482 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.036785:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55482 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.036792:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760743.036798:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55482 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.036807:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55482 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.036829:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55482 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.036838:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55482 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21188608->21192703] (req 21188608->21192703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.036944:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508343424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39039us (39317us total) trans 4294972493 rc 0/0 00000100:00100000:1.0:1789760743.036956:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5205 00000100:00100000:1.0:1789760743.036967:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722058653839 time 722102595503 state 0x2 ref 0 00000100:00100000:3.0:1789760743.041971:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760743.042597:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508343552 00000100:00100000:0.0:1789760743.042607:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508343552 00000100:00100000:2.0:1789760743.042816:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722058653839 time 722102595503 state 0x0 ref 1 00000100:00100000:2.0:1789760743.042833:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722102855919 time 722102595503 state 0x1 ref 1 00000100:00100000:2.0:1789760743.042857:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722102855919 time 722147567735 state 0x0 ref 1 00000100:00100000:2.0:1789760743.042869:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508343552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760743.042915:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21192704, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:2.0:1789760743.042925:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55489 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.042942:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760743.042947:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.042954:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55489 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.042970:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55489 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760743.045818:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760743.064820:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972494 is committed 00002000:00010000:0.0:1789760743.065179:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21192704 -> 21196800 00010000:00010000:0.0:1789760743.065194:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55489 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.065208:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55489 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.065218:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55489 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.065225:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760743.065231:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55489 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.065240:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55489 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.065263:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55489 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.065270:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55489 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21192704->21196799] (req 21192704->21196799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.065536:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508343552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22663us (23569us total) trans 4294972494 rc 0/0 00000100:00100000:0.0:1789760743.065547:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5206 00000100:00100000:0.0:1789760743.065558:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722102855919 time 722147567735 state 0x2 ref 0 00000100:00100000:3.0:1789760743.068702:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760743.068822:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508343680 00000100:00100000:0.0:1789760743.068833:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508343680 00000100:00100000:0.0:1789760743.068893:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722102855919 time 722147567735 state 0x0 ref 1 00000100:00100000:0.0:1789760743.068912:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722147828151 time 722147567735 state 0x1 ref 1 00000100:00100000:0.0:1789760743.068946:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722147828151 time 722173656090 state 0x0 ref 1 00000100:00100000:0.0:1789760743.068959:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508343680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760743.069012:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21196800, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:0.0:1789760743.069022:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55490 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.069040:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760743.069045:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.069052:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55490 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.069079:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55490 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.102718:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972495 is committed 00002000:00010000:0.0:1789760743.102828:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21196800 -> 21200896 00010000:00010000:0.0:1789760743.102845:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55490 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.102857:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55490 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.102866:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55490 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.102873:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760743.102877:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55490 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.102886:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55490 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.102912:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55490 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.102920:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55490 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21196800->21200895] (req 21196800->21200895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.103012:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508343680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34050us (34315us total) trans 4294972495 rc 0/0 00000100:00100000:0.0:1789760743.103021:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5207 00000100:00100000:0.0:1789760743.103032:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722147828151 time 722173656090 state 0x2 ref 0 00000100:00100000:1.0:1789760743.113633:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760743.114227:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508343808 00000100:00100000:3.0:1789760743.114239:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508343808 00000100:00100000:3.0:1789760743.114339:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722147828151 time 722173656090 state 0x0 ref 1 00000100:00100000:3.0:1789760743.114358:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722173916506 time 722173656090 state 0x1 ref 1 00000100:00100000:3.0:1789760743.114391:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722173916506 time 722219101484 state 0x0 ref 1 00000100:00100000:3.0:1789760743.114405:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508343808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760743.114579:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21200896, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:2.0:1789760743.114591:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55497 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.114615:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760743.114620:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.114629:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55497 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.114644:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55497 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.137083:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972496 is committed 00002000:00010000:0.0:1789760743.137231:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21200896 -> 21204992 00010000:00010000:0.0:1789760743.137248:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55497 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.137261:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55497 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.137271:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55497 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.137280:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760743.137311:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55497 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.137327:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55497 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.137350:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55497 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.137358:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55497 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21200896->21204991] (req 21200896->21204991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.137445:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508343808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23038us (23817us total) trans 4294972496 rc 0/0 00000100:00100000:0.0:1789760743.137454:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5208 00000100:00100000:0.0:1789760743.137465:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722173916506 time 722219101484 state 0x2 ref 0 00000100:00100000:1.0:1789760743.143446:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760743.144265:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508343936 00000100:00100000:0.0:1789760743.144277:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508343936 00000100:00100000:0.0:1789760743.144352:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722173916506 time 722219101484 state 0x0 ref 1 00000100:00100000:0.0:1789760743.144371:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722219361900 time 722219101484 state 0x1 ref 1 00000100:00100000:0.0:1789760743.144393:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722219361900 time 722249103573 state 0x0 ref 1 00000100:00100000:0.0:1789760743.144406:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508343936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760743.144456:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21204992, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:0.0:1789760743.144466:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5549e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.144484:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760743.144489:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.144496:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5549e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.144513:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5549e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.168312:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760743.183424:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972497 is committed 00002000:00010000:3.0:1789760743.183515:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21204992 -> 21209088 00010000:00010000:3.0:1789760743.183533:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5549e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.183548:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5549e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.183558:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5549e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.183566:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760743.183573:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5549e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.183661:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5549e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.183687:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5549e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.183696:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5549e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21204992->21209087] (req 21204992->21209087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.183799:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508343936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39389us (40358us total) trans 4294972497 rc 0/0 00000100:00100000:3.0:1789760743.183810:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5209 00000100:00100000:3.0:1789760743.183820:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722219361900 time 722249103573 state 0x2 ref 0 00000100:00100000:3.0:1789760743.197744:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760743.197896:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508344064 00000100:00100000:0.0:1789760743.197908:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508344064 00000100:00100000:0.0:1789760743.197968:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722219361900 time 722249103573 state 0x0 ref 1 00000100:00100000:0.0:1789760743.197995:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722249363989 time 722249103573 state 0x1 ref 1 00000100:00100000:0.0:1789760743.198035:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722249363989 time 722302744923 state 0x0 ref 1 00000100:00100000:0.0:1789760743.198050:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508344064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760743.198267:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21209088, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:0.0:1789760743.198277:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe554a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.198330:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760743.198335:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.198342:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe554a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.198363:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe554a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.266532:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972498 is committed 00002000:00010000:2.0:1789760743.266709:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21209088 -> 21213184 00010000:00010000:2.0:1789760743.266728:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe554a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.266742:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe554a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.266751:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe554a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.266759:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760743.266764:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe554a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.266774:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe554a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.266805:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe554a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.266814:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe554a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21209088->21213183] (req 21209088->21213183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.266947:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508344064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 68895us (69208us total) trans 4294972498 rc 0/0 00000100:00100000:2.0:1789760743.266958:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5210 00000100:00100000:2.0:1789760743.266970:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722249363989 time 722302744923 state 0x2 ref 0 00000100:00100000:0.0:1789760743.274408:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760743.274507:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508344192 00000100:00100000:2.0:1789760743.274517:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508344192 00000100:00100000:2.0:1789760743.274586:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722249363989 time 722302744923 state 0x0 ref 1 00000100:00100000:2.0:1789760743.274604:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722303005339 time 722302744923 state 0x1 ref 1 00000100:00100000:2.0:1789760743.274631:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722303005339 time 722379341137 state 0x0 ref 1 00000100:00100000:2.0:1789760743.274644:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508344192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760743.274698:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21213184, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:2.0:1789760743.274708:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe554ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.274726:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760743.274731:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.274738:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe554ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.274755:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe554ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760743.314901:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21213184 -> 21217280 00010000:00010000:0.0:1789760743.314922:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe554ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.314937:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe554ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.314946:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe554ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.314953:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760743.314958:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe554ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.314968:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe554ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.315003:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe554ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.315010:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe554ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21213184->21217279] (req 21213184->21217279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760743.315119:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508344192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40471us (40714us total) trans 4294972499 rc 0/0 00000100:00100000:0.0:1789760743.315131:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5211 00000100:00100000:0.0:1789760743.315141:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722303005339 time 722379341137 state 0x2 ref 0 00000001:00080000:3.0:1789760743.316333:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972499 is committed 00000100:00100000:1.0:1789760743.326896:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760743.326983:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508344320 00000100:00100000:3.0:1789760743.326995:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508344320 00000100:00100000:3.0:1789760743.327056:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722303005339 time 722379341137 state 0x0 ref 1 00000100:00100000:3.0:1789760743.327084:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722379601553 time 722379341137 state 0x1 ref 1 00000100:00100000:3.0:1789760743.327113:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722379601553 time 722431823921 state 0x0 ref 1 00000100:00100000:3.0:1789760743.327125:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508344320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760743.327183:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21217280, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:3.0:1789760743.327192:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe554b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.327210:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.327215:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.327222:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe554b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.327238:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe554b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.374786:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972500 is committed 00002000:00010000:2.0:1789760743.374958:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21217280 -> 21221376 00010000:00010000:2.0:1789760743.374989:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe554b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.375002:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe554b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.375011:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe554b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.375018:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760743.375023:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe554b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.375033:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe554b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.375067:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe554b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.375081:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe554b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21217280->21221375] (req 21217280->21221375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.375180:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508344320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48051us (48288us total) trans 4294972500 rc 0/0 00000100:00100000:2.0:1789760743.375191:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5212 00000100:00100000:2.0:1789760743.375201:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722379601553 time 722431823921 state 0x2 ref 0 00000100:00100000:2.0:1789760743.379651:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760743.379732:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508344448 00000100:00100000:3.0:1789760743.379743:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508344448 00000100:00100000:3.0:1789760743.379803:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722379601553 time 722431823921 state 0x0 ref 1 00000100:00100000:3.0:1789760743.379824:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722432084337 time 722431823921 state 0x1 ref 1 00000100:00100000:3.0:1789760743.379857:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722432084337 time 722484567482 state 0x0 ref 1 00000100:00100000:3.0:1789760743.379870:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508344448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760743.379923:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21221376, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:3.0:1789760743.379934:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.379951:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.379956:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.379963:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.379979:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760743.402124:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972501 is committed 00002000:00010000:3.0:1789760743.402261:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21221376 -> 21225472 00010000:00010000:3.0:1789760743.402277:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.402306:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.402315:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.402326:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760743.402331:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.402340:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.402363:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.402371:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21221376->21225471] (req 21221376->21225471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.402598:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508344448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22724us (22950us total) trans 4294972501 rc 0/0 00000100:00100000:3.0:1789760743.403171:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5213 00000100:00100000:3.0:1789760743.403186:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722432084337 time 722484567482 state 0x2 ref 0 00000100:00100000:0.0:1789760743.407503:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760743.407606:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508344576 00000100:00100000:2.0:1789760743.407619:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508344576 00000100:00100000:2.0:1789760743.407679:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722432084337 time 722484567482 state 0x0 ref 1 00000100:00100000:2.0:1789760743.407695:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722484827898 time 722484567482 state 0x1 ref 1 00000100:00100000:2.0:1789760743.407729:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722484827898 time 722512439132 state 0x0 ref 1 00000100:00100000:2.0:1789760743.407743:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508344576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760743.407796:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21225472, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:2.0:1789760743.407807:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe554c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.407824:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760743.407829:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.407836:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe554c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.407853:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe554c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.457984:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972502 is committed 00002000:00010000:1.0:1789760743.458242:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21225472 -> 21229568 00010000:00010000:1.0:1789760743.458261:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe554c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.458275:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe554c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.458302:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe554c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.458310:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760743.458315:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe554c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.458324:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe554c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.458349:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe554c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.458356:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe554c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21225472->21229567] (req 21225472->21229567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.458535:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508344576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50789us (51035us total) trans 4294972502 rc 0/0 00000100:00100000:1.0:1789760743.458546:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5214 00000100:00100000:1.0:1789760743.458557:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722484827898 time 722512439132 state 0x2 ref 0 00000100:00100000:1.0:1789760743.463757:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760743.464543:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508344704 00000100:00100000:2.0:1789760743.464557:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508344704 00000100:00100000:2.0:1789760743.464617:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722484827898 time 722512439132 state 0x0 ref 1 00000100:00100000:2.0:1789760743.464633:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722512699548 time 722512439132 state 0x1 ref 1 00000100:00100000:2.0:1789760743.464665:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722512699548 time 722569375336 state 0x0 ref 1 00000100:00100000:2.0:1789760743.464678:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d47050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508344704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760743.464725:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21229568, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:2.0:1789760743.464736:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe554c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.464753:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760743.464757:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.464764:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe554c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.464780:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe554c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.491424:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972503 is committed 00002000:00010000:3.0:1789760743.491675:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21229568 -> 21233664 00010000:00010000:3.0:1789760743.491695:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe554c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.491709:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe554c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.491723:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe554c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.491730:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760743.491734:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe554c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.491743:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe554c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.491767:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe554c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.491773:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe554c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21229568->21233663] (req 21229568->21233663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.491946:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d47050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508344704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27265us (28366us total) trans 4294972503 rc 0/0 00000100:00100000:3.0:1789760743.491959:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5215 00000100:00100000:3.0:1789760743.491970:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722512699548 time 722569375336 state 0x2 ref 0 00000100:00100000:2.0:1789760743.496709:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760743.497685:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508344832 00000100:00100000:3.0:1789760743.497699:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508344832 00000100:00100000:3.0:1789760743.497775:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722512699548 time 722569375336 state 0x0 ref 1 00000100:00100000:3.0:1789760743.497793:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722569635752 time 722569375336 state 0x1 ref 1 00000100:00100000:3.0:1789760743.497829:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722569635752 time 722602539099 state 0x0 ref 1 00000100:00100000:3.0:1789760743.497843:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508344832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760743.497899:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21233664, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0xa807 00010000:00010000:3.0:1789760743.497914:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.497931:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.497935:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.497942:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.497957:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.542544:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972504 is committed 00002000:00010000:3.0:1789760743.546724:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21233664 -> 21237760 00002000:00010000:3.0:1789760743.546733:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760743 -> 1789760744 00002000:00010000:3.0:1789760743.546736:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760743 -> 1789760744 00010000:00010000:3.0:1789760743.546749:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.546763:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.546772:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.546780:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760743.546785:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.546794:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.546816:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.546823:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21233664->21237759] (req 21233664->21237759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.546972:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508344832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49126us (50268us total) trans 4294972504 rc 0/0 00000100:00100000:3.0:1789760743.546991:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5216 00000100:00100000:3.0:1789760743.547002:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722569635752 time 722602539099 state 0x2 ref 0 00000100:00100000:0.0:1789760743.558394:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760743.558651:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508344960 00000100:00100000:1.0:1789760743.558663:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508344960 00000100:00100000:1.0:1789760743.558720:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722569635752 time 722602539099 state 0x0 ref 1 00000100:00100000:1.0:1789760743.558735:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722602799515 time 722602539099 state 0x1 ref 1 00000100:00100000:1.0:1789760743.558813:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722602799515 time 722663522880 state 0x0 ref 1 00000100:00100000:1.0:1789760743.558827:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508344960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760743.558881:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21237760, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:1.0:1789760743.558891:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe554d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.558913:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760743.558917:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.558924:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe554d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.558938:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe554d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760743.588848:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21237760 -> 21241856 00010000:00010000:2.0:1789760743.588870:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe554d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.588884:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe554d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.588894:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe554d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.588902:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760743.588907:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe554d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.588917:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe554d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.588940:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe554d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.588948:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe554d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21237760->21241855] (req 21237760->21241855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.589044:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508344960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30214us (30654us total) trans 4294972505 rc 0/0 00000100:00100000:2.0:1789760743.589055:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5217 00000100:00100000:2.0:1789760743.589076:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722602799515 time 722663522880 state 0x2 ref 0 00000001:00080000:3.0:1789760743.591896:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972505 is committed 00000100:00100000:1.0:1789760743.592777:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760743.592852:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508345088 00000100:00100000:3.0:1789760743.592862:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508345088 00000100:00100000:3.0:1789760743.592921:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722602799515 time 722663522880 state 0x0 ref 1 00000100:00100000:3.0:1789760743.592939:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722663783296 time 722663522880 state 0x1 ref 1 00000100:00100000:3.0:1789760743.592971:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722663783296 time 722697680964 state 0x0 ref 1 00000100:00100000:3.0:1789760743.592984:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508345088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760743.593035:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21241856, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:3.0:1789760743.593044:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.593071:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.593076:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.593083:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.593099:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.627814:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21241856 -> 21245952 00010000:00010000:3.0:1789760743.627835:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.627849:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.627857:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.627864:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760743.627869:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.627884:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.627908:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.627916:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe554dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21241856->21245951] (req 21241856->21245951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.628025:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508345088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35037us (35248us total) trans 4294972506 rc 0/0 00000100:00100000:3.0:1789760743.628037:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5218 00000100:00100000:3.0:1789760743.628048:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722663783296 time 722697680964 state 0x2 ref 0 00000001:00080000:1.0:1789760743.631236:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972506 is committed 00000100:00100000:1.0:1789760743.632470:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760743.632722:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508345216 00000100:00100000:3.0:1789760743.632732:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508345216 00000100:00100000:3.0:1789760743.632788:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722663783296 time 722697680964 state 0x0 ref 1 00000100:00100000:3.0:1789760743.632804:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722697941380 time 722697680964 state 0x1 ref 1 00000100:00100000:3.0:1789760743.632835:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722697941380 time 722737545785 state 0x0 ref 1 00000100:00100000:3.0:1789760743.632849:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508345216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760743.632899:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21245952, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:3.0:1789760743.632908:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.632925:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.632930:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.632937:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.632951:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.658494:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972507 is committed 00002000:00010000:3.0:1789760743.658648:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21245952 -> 21250048 00010000:00010000:3.0:1789760743.658665:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.658677:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.658685:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.658692:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760743.658697:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.658707:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.658731:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.658739:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21245952->21250047] (req 21245952->21250047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.658834:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508345216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25983us (26366us total) trans 4294972507 rc 0/0 00000100:00100000:3.0:1789760743.658846:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5219 00000100:00100000:3.0:1789760743.658855:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722697941380 time 722737545785 state 0x2 ref 0 00000100:00100000:2.0:1789760743.665222:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760743.665390:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508345344 00000100:00100000:1.0:1789760743.665401:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508345344 00000100:00100000:1.0:1789760743.665458:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722697941380 time 722737545785 state 0x0 ref 1 00000100:00100000:1.0:1789760743.665486:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722737806201 time 722737545785 state 0x1 ref 1 00000100:00100000:2.0:1789760743.665541:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722737806201 time 722770249541 state 0x0 ref 1 00000100:00100000:2.0:1789760743.665556:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508345344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760743.665606:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21250048, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:2.0:1789760743.665618:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe554eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.665635:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760743.665640:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.665647:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe554eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.665663:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe554eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760743.702942:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972508 is committed 00002000:00010000:1.0:1789760743.703131:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21250048 -> 21254144 00010000:00010000:1.0:1789760743.703148:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe554eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.703162:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe554eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.703171:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe554eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.703185:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760743.703190:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe554eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.703200:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe554eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.703223:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe554eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.703230:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe554eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21250048->21254143] (req 21250048->21254143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760743.703369:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508345344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37811us (38151us total) trans 4294972508 rc 0/0 00000100:00100000:1.0:1789760743.703381:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5220 00000100:00100000:1.0:1789760743.703393:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722737806201 time 722770249541 state 0x2 ref 0 00000100:00100000:1.0:1789760743.709597:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760743.709670:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508345472 00000100:00100000:3.0:1789760743.709681:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508345472 00000100:00100000:3.0:1789760743.709755:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722737806201 time 722770249541 state 0x0 ref 1 00000100:00100000:3.0:1789760743.709773:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722770509957 time 722770249541 state 0x1 ref 1 00000100:00100000:3.0:1789760743.709811:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722770509957 time 722814520926 state 0x0 ref 1 00000100:00100000:3.0:1789760743.709824:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508345472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760743.709886:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21254144, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:3.0:1789760743.709897:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.709914:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.709919:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.709927:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.709959:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.755606:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972509 is committed 00002000:00010000:3.0:1789760743.755710:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21254144 -> 21258240 00010000:00010000:3.0:1789760743.755728:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.755741:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.755750:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.755757:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760743.755762:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.755772:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.755797:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.755806:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21254144->21258239] (req 21254144->21258239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.755915:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508345472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46086us (46320us total) trans 4294972509 rc 0/0 00000100:00100000:3.0:1789760743.755925:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5221 00000100:00100000:3.0:1789760743.755936:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722770509957 time 722814520926 state 0x2 ref 0 00000100:00100000:1.0:1789760743.764538:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760743.764643:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508345600 00000100:00100000:3.0:1789760743.764654:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508345600 00000100:00100000:3.0:1789760743.764711:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722770509957 time 722814520926 state 0x0 ref 1 00000100:00100000:3.0:1789760743.764727:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722814781342 time 722814520926 state 0x1 ref 1 00000100:00100000:3.0:1789760743.764757:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722814781342 time 722869467336 state 0x0 ref 1 00000100:00100000:3.0:1789760743.764771:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508345600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760743.764831:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21258240, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:3.0:1789760743.764841:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.764859:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:3.0:1789760743.764864:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.764871:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.764886:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760743.800910:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972510 is committed 00002000:00010000:2.0:1789760743.801121:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21258240 -> 21262336 00010000:00010000:2.0:1789760743.801142:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.801156:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.801166:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.801175:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760743.801180:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.801189:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.801212:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.801220:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe554f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21258240->21262335] (req 21258240->21262335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.801570:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508345600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36796us (37038us total) trans 4294972510 rc 0/0 00000100:00100000:2.0:1789760743.801582:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5222 00000100:00100000:2.0:1789760743.801592:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722814781342 time 722869467336 state 0x2 ref 0 00000100:00100000:2.0:1789760743.806724:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760743.806798:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508345728 00000100:00100000:1.0:1789760743.806807:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508345728 00000100:00100000:1.0:1789760743.806860:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722814781342 time 722869467336 state 0x0 ref 1 00000100:00100000:1.0:1789760743.806876:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722869727752 time 722869467336 state 0x1 ref 1 00000100:00100000:1.0:1789760743.806906:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722869727752 time 722911616956 state 0x0 ref 1 00000100:00100000:1.0:1789760743.806917:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508345728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760743.806962:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21262336, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:1.0:1789760743.806972:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55500 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.806997:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760743.807001:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760743.807008:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55500 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760743.807022:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55500 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.844656:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972511 is committed 00002000:00010000:2.0:1789760743.844875:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21262336 -> 21266432 00010000:00010000:2.0:1789760743.844892:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55500 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.844905:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55500 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.844920:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55500 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.844927:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760743.844931:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55500 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.844946:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55500 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.844970:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55500 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.844978:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55500 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21262336->21266431] (req 21262336->21266431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.845076:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508345728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38154us (38357us total) trans 4294972511 rc 0/0 00000100:00100000:2.0:1789760743.845087:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5223 00000100:00100000:2.0:1789760743.845098:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722869727752 time 722911616956 state 0x2 ref 0 00000100:00100000:3.0:1789760743.847801:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760743.848098:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508345856 00000100:00100000:2.0:1789760743.848109:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508345856 00000100:00100000:2.0:1789760743.848166:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722869727752 time 722911616956 state 0x0 ref 1 00000100:00100000:2.0:1789760743.848236:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722911877372 time 722911616956 state 0x1 ref 1 00000100:00100000:2.0:1789760743.848272:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722911877372 time 722952981758 state 0x0 ref 1 00000100:00100000:2.0:1789760743.848323:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508345856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760743.848369:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21266432, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:2.0:1789760743.848376:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55507 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.848390:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760743.848393:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.848398:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55507 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.848411:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55507 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.904991:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972512 is committed 00002000:00010000:2.0:1789760743.905161:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21266432 -> 21270528 00010000:00010000:2.0:1789760743.905184:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55507 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.905197:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55507 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.905207:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55507 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.905214:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760743.905220:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55507 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.905228:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55507 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.905252:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55507 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.905259:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55507 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21266432->21270527] (req 21266432->21270527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760743.905393:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508345856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57103us (57596us total) trans 4294972512 rc 0/0 00000100:00100000:2.0:1789760743.905402:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5224 00000100:00100000:2.0:1789760743.905413:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722911877372 time 722952981758 state 0x2 ref 0 00000100:00100000:3.0:1789760743.920030:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760743.920515:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508345984 00000100:00100000:2.0:1789760743.920525:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508345984 00000100:00100000:2.0:1789760743.920628:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722911877372 time 722952981758 state 0x0 ref 1 00000100:00100000:2.0:1789760743.920647:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722953242174 time 722952981758 state 0x1 ref 1 00000100:00100000:0.0:1789760743.924050:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722953242174 time 723028744970 state 0x0 ref 1 00000100:00100000:0.0:1789760743.924109:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508345984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760743.924183:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21270528, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:0.0:1789760743.924193:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5550e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.924219:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760743.924223:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760743.924231:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5550e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760743.924248:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5550e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.975910:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21270528 -> 21274624 00010000:00010000:3.0:1789760743.975934:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5550e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.975948:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5550e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.975958:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5550e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.975966:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760743.975971:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5550e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.975981:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5550e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.976004:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5550e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.976012:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5550e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21270528->21274623] (req 21270528->21274623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.976122:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508345984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52033us (56096us total) trans 4294972513 rc 0/0 00000100:00100000:3.0:1789760743.976133:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5225 00000100:00100000:3.0:1789760743.976144:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722953242174 time 723028744970 state 0x2 ref 0 00000001:00080000:1.0:1789760743.976710:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972513 is committed 00000100:00100000:3.0:1789760743.983172:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760743.983331:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508346112 00000100:00100000:2.0:1789760743.983343:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508346112 00000100:00100000:2.0:1789760743.983565:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 722953242174 time 723028744970 state 0x0 ref 1 00000100:00100000:2.0:1789760743.983585:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723029005386 time 723028744970 state 0x1 ref 1 00000100:00100000:2.0:1789760743.983618:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723029005386 time 723088328271 state 0x0 ref 1 00000100:00100000:2.0:1789760743.983632:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508346112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760743.983691:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21274624, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:2.0:1789760743.983701:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55515 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.983718:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760743.983723:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760743.983731:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55515 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760743.983746:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55515 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.030704:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972514 is committed 00002000:00010000:2.0:1789760744.030770:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21274624 -> 21278720 00010000:00010000:2.0:1789760744.030787:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55515 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.030799:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55515 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.030809:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55515 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.030822:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760744.030828:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55515 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.030837:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55515 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.030859:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55515 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.030867:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55515 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21274624->21278719] (req 21274624->21278719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.030960:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508346112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47325us (47792us total) trans 4294972514 rc 0/0 00000100:00100000:2.0:1789760744.030971:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5226 00000100:00100000:2.0:1789760744.030981:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723029005386 time 723088328271 state 0x2 ref 0 00000100:00100000:2.0:1789760744.047460:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760744.047609:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508346240 00000100:00100000:3.0:1789760744.047619:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508346240 00000100:00100000:3.0:1789760744.047683:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723029005386 time 723088328271 state 0x0 ref 1 00000100:00100000:3.0:1789760744.047699:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723088588687 time 723088328271 state 0x1 ref 1 00000100:00100000:3.0:1789760744.047734:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723088588687 time 723152444541 state 0x0 ref 1 00000100:00100000:3.0:1789760744.047746:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508346240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760744.047796:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21278720, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:3.0:1789760744.047805:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5551c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.047823:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760744.047828:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.047834:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5551c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.047849:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5551c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.083180:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972515 is committed 00002000:00010000:3.0:1789760744.083319:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21278720 -> 21282816 00010000:00010000:3.0:1789760744.083336:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5551c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.083348:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5551c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.083357:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5551c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.083364:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760744.083369:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5551c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.083378:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5551c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.083618:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5551c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.083628:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5551c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21278720->21282815] (req 21278720->21282815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.083827:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508346240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36077us (36371us total) trans 4294972515 rc 0/0 00000100:00100000:2.0:1789760744.083838:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5227 00000100:00100000:2.0:1789760744.083848:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723088588687 time 723152444541 state 0x2 ref 0 00000100:00100000:2.0:1789760744.087947:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760744.088600:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508346368 00000100:00100000:3.0:1789760744.088611:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508346368 00000100:00100000:0.0:1789760744.088744:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723088588687 time 723152444541 state 0x0 ref 1 00000100:00100000:0.0:1789760744.088762:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723152704957 time 723152444541 state 0x1 ref 1 00000100:00100000:0.0:1789760744.088837:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723152704957 time 723193546224 state 0x0 ref 1 00000100:00100000:0.0:1789760744.088855:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508346368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760744.088907:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21282816, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:0.0:1789760744.088917:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55523 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.088934:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.088939:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.088946:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55523 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.088962:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55523 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760744.130031:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21282816 -> 21286912 00010000:00010000:2.0:1789760744.130074:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55523 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.130089:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55523 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.130108:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55523 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.130117:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760744.130122:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55523 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.130131:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55523 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.130161:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55523 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.130169:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55523 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21282816->21286911] (req 21282816->21286911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.130314:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508346368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41430us (42342us total) trans 4294972516 rc 0/0 00000100:00100000:2.0:1789760744.130329:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5228 00000100:00100000:2.0:1789760744.130338:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723152704957 time 723193546224 state 0x2 ref 0 00000001:00080000:3.0:1789760744.130438:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972516 is committed 00000100:00100000:3.0:1789760744.136618:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760744.136771:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508346496 00000100:00100000:2.0:1789760744.136784:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508346496 00000100:00100000:2.0:1789760744.136847:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723152704957 time 723193546224 state 0x0 ref 1 00000100:00100000:2.0:1789760744.136865:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723193806640 time 723193546224 state 0x1 ref 1 00000100:00100000:2.0:1789760744.136916:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723193806640 time 723241625896 state 0x0 ref 1 00000100:00100000:2.0:1789760744.136930:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508346496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760744.136980:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21286912, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:2.0:1789760744.136990:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5552a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.137009:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760744.137014:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.137022:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5552a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.137039:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5552a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.190659:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760744.201992:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21286912 -> 21291008 00010000:00010000:3.0:1789760744.202014:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5552a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.202029:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5552a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.202038:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5552a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.202045:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760744.202051:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5552a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.202069:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5552a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.202091:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5552a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.202099:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5552a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21286912->21291007] (req 21286912->21291007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.202225:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508346496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 65291us (65612us total) trans 4294972517 rc 0/0 00000100:00100000:3.0:1789760744.202235:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5229 00000100:00100000:3.0:1789760744.202253:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723193806640 time 723241625896 state 0x2 ref 0 00000001:00080000:3.0:1789760744.202322:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972517 is committed 00000100:00100000:1.0:1789760744.210547:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760744.210673:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508346624 00000100:00100000:2.0:1789760744.210685:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508346624 00000100:00100000:2.0:1789760744.210746:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723193806640 time 723241625896 state 0x0 ref 1 00000100:00100000:2.0:1789760744.210764:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723241886312 time 723241625896 state 0x1 ref 1 00000100:00100000:2.0:1789760744.210831:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723241886312 time 723315541249 state 0x0 ref 1 00000100:00100000:2.0:1789760744.210846:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508346624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760744.210898:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21291008, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:2.0:1789760744.210908:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55531 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.210925:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760744.210931:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.210937:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55531 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.210952:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55531 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760744.232483:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21291008 -> 21295104 00010000:00010000:2.0:1789760744.232504:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55531 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.232517:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55531 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.232526:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55531 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.232534:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760744.232539:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55531 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.232549:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55531 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.232572:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55531 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.232581:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55531 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21291008->21295103] (req 21291008->21295103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.232674:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508346624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 21824us (22131us total) trans 4294972518 rc 0/0 00000100:00100000:2.0:1789760744.232683:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5230 00000100:00100000:2.0:1789760744.232693:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723241886312 time 723315541249 state 0x2 ref 0 00000001:00080000:3.0:1789760744.232816:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972518 is committed 00000100:00100000:2.0:1789760744.238228:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760744.238328:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508346752 00000100:00100000:0.0:1789760744.238338:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508346752 00000100:00100000:0.0:1789760744.238389:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723241886312 time 723315541249 state 0x0 ref 1 00000100:00100000:0.0:1789760744.238403:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723315801665 time 723315541249 state 0x1 ref 1 00000100:00100000:0.0:1789760744.238428:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723315801665 time 723343138788 state 0x0 ref 1 00000100:00100000:0.0:1789760744.238439:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508346752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760744.238490:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21295104, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:0.0:1789760744.238500:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55538 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.238518:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.238523:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.238531:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55538 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.238547:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55538 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760744.269985:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972519 is committed 00002000:00010000:1.0:1789760744.270032:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21295104 -> 21299200 00010000:00010000:1.0:1789760744.270049:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55538 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.270074:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55538 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.270085:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55538 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.270094:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760744.270099:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55538 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.270113:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55538 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.270136:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55538 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.270144:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55538 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21295104->21299199] (req 21295104->21299199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.270237:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508346752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31793us (32012us total) trans 4294972519 rc 0/0 00000100:00100000:1.0:1789760744.270248:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5231 00000100:00100000:1.0:1789760744.270258:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723315801665 time 723343138788 state 0x2 ref 0 00000100:00100000:0.0:1789760744.273654:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760744.273758:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508346880 00000100:00100000:3.0:1789760744.273768:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508346880 00000100:00100000:3.0:1789760744.273825:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723315801665 time 723343138788 state 0x0 ref 1 00000100:00100000:3.0:1789760744.273841:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723343399204 time 723343138788 state 0x1 ref 1 00000100:00100000:3.0:1789760744.273908:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723343399204 time 723378617266 state 0x0 ref 1 00000100:00100000:3.0:1789760744.273922:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508346880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760744.273990:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21299200, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:3.0:1789760744.274001:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5553f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.274018:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760744.274023:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.274039:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5553f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.274066:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5553f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.303955:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972520 is committed 00002000:00010000:3.0:1789760744.304161:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21299200 -> 21303296 00010000:00010000:3.0:1789760744.304177:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5553f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.304192:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5553f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.304202:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5553f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.304209:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760744.304215:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5553f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.304225:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5553f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.304249:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5553f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.304257:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5553f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21299200->21303295] (req 21299200->21303295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.304617:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508346880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30691us (30966us total) trans 4294972520 rc 0/0 00000100:00100000:3.0:1789760744.304627:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5232 00000100:00100000:3.0:1789760744.304637:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723343399204 time 723378617266 state 0x2 ref 0 00000100:00100000:2.0:1789760744.313165:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760744.313374:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508347008 00000100:00100000:0.0:1789760744.313384:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508347008 00000100:00100000:0.0:1789760744.313454:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723343399204 time 723378617266 state 0x0 ref 1 00000100:00100000:0.0:1789760744.313473:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723378877682 time 723378617266 state 0x1 ref 1 00000100:00100000:0.0:1789760744.313495:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723378877682 time 723418205831 state 0x0 ref 1 00000100:00100000:0.0:1789760744.313507:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508347008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760744.313559:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21303296, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:0.0:1789760744.313570:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55546 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.313587:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.313591:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.313598:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55546 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.313613:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55546 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.319088:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760744.322120:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048880f8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534070784:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760744.322229:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760744.324424:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534070784 00000100:00100000:2.0:1789760744.324436:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534070784 00000100:00100000:2.0:1789760744.324498:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:2.0:1789760744.324508:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534070784:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760744.324552:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789760744.324629:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534070784:12345-0@lo:41:osp-pre-1-0.0 Request processed in 122us (2399us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760744.324636:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00100000:2.0:1789760744.324686:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048880f8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534070784:0@lo:41:osp-pre-1-0.0 00000001:00080000:3.0:1789760744.351915:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972521 is committed 00002000:00010000:0.0:1789760744.352119:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21303296 -> 21307392 00010000:00010000:0.0:1789760744.352135:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55546 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.352148:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55546 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.352158:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55546 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.352166:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760744.352172:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55546 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.352182:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55546 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.352206:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55546 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.352216:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55546 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21303296->21307391] (req 21303296->21307391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.352386:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508347008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38875us (39225us total) trans 4294972521 rc 0/0 00000100:00100000:0.0:1789760744.352396:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5233 00000100:00100000:0.0:1789760744.352415:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723378877682 time 723418205831 state 0x2 ref 0 00000100:00100000:2.0:1789760744.356452:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760744.356579:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508347136 00000100:00100000:1.0:1789760744.356589:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508347136 00000100:00100000:1.0:1789760744.356642:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723378877682 time 723418205831 state 0x0 ref 1 00000100:00100000:1.0:1789760744.356658:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723418466247 time 723418205831 state 0x1 ref 1 00000100:00100000:1.0:1789760744.356688:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723418466247 time 723461399112 state 0x0 ref 1 00000100:00100000:1.0:1789760744.356700:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508347136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760744.356746:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21307392, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:1.0:1789760744.356756:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5554d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.356772:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760744.356776:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.356782:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5554d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.356797:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5554d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.394786:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972522 is committed 00002000:00010000:2.0:1789760744.394921:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21307392 -> 21311488 00010000:00010000:2.0:1789760744.394940:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5554d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.394954:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5554d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.394964:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5554d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.394972:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760744.394978:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5554d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.394991:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5554d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.395014:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5554d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.395028:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5554d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21307392->21311487] (req 21307392->21311487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.395616:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508347136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38909us (39175us total) trans 4294972522 rc 0/0 00000100:00100000:2.0:1789760744.395627:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5234 00000100:00100000:2.0:1789760744.395637:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723418466247 time 723461399112 state 0x2 ref 0 00000100:00100000:1.0:1789760744.401900:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760744.405583:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508347264 00000100:00100000:3.0:1789760744.405599:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508347264 00000100:00100000:3.0:1789760744.405673:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723418466247 time 723461399112 state 0x0 ref 1 00000100:00100000:3.0:1789760744.405692:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723461659528 time 723461399112 state 0x1 ref 1 00000100:00100000:3.0:1789760744.405791:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723461659528 time 723510500068 state 0x0 ref 1 00000100:00100000:3.0:1789760744.405807:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508347264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760744.405860:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21311488, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:3.0:1789760744.405870:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55554 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.405889:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760744.405893:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.405900:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55554 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.405916:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55554 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760744.431326:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21311488 -> 21315584 00010000:00010000:0.0:1789760744.431346:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55554 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.431358:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55554 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.431368:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55554 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.431376:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760744.431381:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55554 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.431397:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55554 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.431539:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55554 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.431547:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55554 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21311488->21315583] (req 21311488->21315583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.431646:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508347264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25836us (29750us total) trans 4294972523 rc 0/0 00000100:00100000:0.0:1789760744.431657:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5235 00000100:00100000:0.0:1789760744.431667:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723461659528 time 723510500068 state 0x2 ref 0 00000001:00080000:2.0:1789760744.433606:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972523 is committed 00000100:00100000:0.0:1789760744.436060:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760744.436575:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508347392 00000100:00100000:2.0:1789760744.436588:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508347392 00000100:00100000:2.0:1789760744.436650:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723461659528 time 723510500068 state 0x0 ref 1 00000100:00100000:2.0:1789760744.436668:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723510760484 time 723510500068 state 0x1 ref 1 00000100:00100000:2.0:1789760744.436701:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723510760484 time 723541411215 state 0x0 ref 1 00000100:00100000:2.0:1789760744.436715:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508347392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760744.436769:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21315584, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:2.0:1789760744.436779:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5555b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.436796:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760744.436800:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.436808:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5555b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.436823:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5555b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.470163:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972524 is committed 00002000:00010000:3.0:1789760744.470258:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21315584 -> 21319680 00010000:00010000:3.0:1789760744.470274:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5555b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.470318:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5555b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.470332:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5555b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.470340:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760744.470345:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5555b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.470355:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5555b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.470379:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5555b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.470387:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5555b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21315584->21319679] (req 21315584->21319679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.470491:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508347392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33773us (34443us total) trans 4294972524 rc 0/0 00000100:00100000:3.0:1789760744.470502:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5236 00000100:00100000:3.0:1789760744.470512:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723510760484 time 723541411215 state 0x2 ref 0 00000100:00100000:3.0:1789760744.474884:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760744.475111:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508347520 00000100:00100000:2.0:1789760744.475121:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508347520 00000100:00100000:2.0:1789760744.475183:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723510760484 time 723541411215 state 0x0 ref 1 00000100:00100000:2.0:1789760744.475199:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723541671631 time 723541411215 state 0x1 ref 1 00000100:00100000:2.0:1789760744.475274:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723541671631 time 723579984630 state 0x0 ref 1 00000100:00100000:2.0:1789760744.475326:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508347520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760744.475386:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21319680, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0xa807 00010000:00010000:2.0:1789760744.475405:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55562 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.475425:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760744.475430:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.475437:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55562 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.475452:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55562 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.509436:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972525 is committed 00002000:00010000:0.0:1789760744.509510:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21319680 -> 21323776 00002000:00010000:0.0:1789760744.509515:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760744 -> 1789760745 00002000:00010000:0.0:1789760744.509517:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760744 -> 1789760745 00010000:00010000:0.0:1789760744.509532:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55562 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.509544:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55562 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.509560:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55562 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.509568:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760744.509573:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55562 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.509582:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55562 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.509607:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55562 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.509614:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55562 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21319680->21323775] (req 21319680->21323775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760744.509703:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508347520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34414us (34823us total) trans 4294972525 rc 0/0 00000100:00100000:0.0:1789760744.509710:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5237 00000100:00100000:0.0:1789760744.509718:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723541671631 time 723579984630 state 0x2 ref 0 00000100:00100000:0.0:1789760744.516170:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760744.516245:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508347648 00000100:00100000:3.0:1789760744.516256:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508347648 00000100:00100000:3.0:1789760744.516331:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723541671631 time 723579984630 state 0x0 ref 1 00000100:00100000:3.0:1789760744.516350:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723580245046 time 723579984630 state 0x1 ref 1 00000100:00100000:3.0:1789760744.516378:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723580245046 time 723621089287 state 0x0 ref 1 00000100:00100000:3.0:1789760744.516390:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508347648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760744.516454:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21323776, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:3.0:1789760744.516465:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55569 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.516493:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760744.516498:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.516506:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55569 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.516522:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55569 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.545177:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972526 is committed 00002000:00010000:2.0:1789760744.545280:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21323776 -> 21327872 00010000:00010000:2.0:1789760744.545331:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55569 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.545346:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55569 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.545355:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55569 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.545363:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760744.545369:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55569 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.545378:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55569 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.545407:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55569 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.545421:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55569 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21323776->21327871] (req 21323776->21327871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.545672:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508347648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29278us (29505us total) trans 4294972526 rc 0/0 00000100:00100000:2.0:1789760744.545683:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5238 00000100:00100000:2.0:1789760744.545693:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723580245046 time 723621089287 state 0x2 ref 0 00000100:00100000:2.0:1789760744.552722:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760744.552819:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508347776 00000100:00100000:3.0:1789760744.552829:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508347776 00000100:00100000:3.0:1789760744.552892:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723580245046 time 723621089287 state 0x0 ref 1 00000100:00100000:3.0:1789760744.552910:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723621349703 time 723621089287 state 0x1 ref 1 00000100:00100000:3.0:1789760744.552991:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723621349703 time 723657701132 state 0x0 ref 1 00000100:00100000:3.0:1789760744.553004:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508347776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760744.553070:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21327872, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:3.0:1789760744.553080:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55570 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.553099:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760744.553103:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.553109:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55570 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.553125:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55570 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.580609:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972527 is committed 00002000:00010000:3.0:1789760744.580791:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21327872 -> 21331968 00010000:00010000:3.0:1789760744.580808:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55570 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.580820:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55570 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.580829:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55570 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.580837:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760744.580842:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55570 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.580852:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55570 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.580872:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55570 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.580877:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55570 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21327872->21331967] (req 21327872->21331967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.580957:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508347776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27951us (28242us total) trans 4294972527 rc 0/0 00000100:00100000:3.0:1789760744.580964:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5239 00000100:00100000:3.0:1789760744.580975:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723621349703 time 723657701132 state 0x2 ref 0 00000100:00100000:3.0:1789760744.585375:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760744.585468:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508347904 00000100:00100000:0.0:1789760744.585543:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508347904 00000100:00100000:0.0:1789760744.585605:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723621349703 time 723657701132 state 0x0 ref 1 00000100:00100000:0.0:1789760744.585621:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723657961548 time 723657701132 state 0x1 ref 1 00000100:00100000:0.0:1789760744.585648:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723657961548 time 723690359161 state 0x0 ref 1 00000100:00100000:0.0:1789760744.585662:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508347904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760744.585714:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21331968, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:0.0:1789760744.585725:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55577 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.585742:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760744.585747:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760744.585754:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55577 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760744.585769:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55577 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.618959:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972528 is committed 00002000:00010000:1.0:1789760744.619061:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21331968 -> 21336064 00010000:00010000:1.0:1789760744.619079:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55577 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.619092:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55577 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.619102:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55577 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.619115:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760744.619121:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55577 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.619131:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55577 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.619154:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55577 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.619163:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55577 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21331968->21336063] (req 21331968->21336063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.619377:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508347904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33712us (34006us total) trans 4294972528 rc 0/0 00000100:00100000:1.0:1789760744.619387:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5240 00000100:00100000:1.0:1789760744.619774:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723657961548 time 723690359161 state 0x2 ref 0 00000100:00100000:0.0:1789760744.623589:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760744.623717:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508348032 00000100:00100000:3.0:1789760744.623726:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508348032 00000100:00100000:3.0:1789760744.623783:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723657961548 time 723690359161 state 0x0 ref 1 00000100:00100000:3.0:1789760744.623800:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723690619577 time 723690359161 state 0x1 ref 1 00000100:00100000:3.0:1789760744.623829:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723690619577 time 723728539068 state 0x0 ref 1 00000100:00100000:3.0:1789760744.623841:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508348032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760744.623888:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21336064, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:3.0:1789760744.623898:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5557e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.623922:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760744.623927:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.623934:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5557e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.623950:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5557e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760744.661957:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972529 is committed 00002000:00010000:3.0:1789760744.662118:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21336064 -> 21340160 00010000:00010000:3.0:1789760744.662135:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5557e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.662148:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5557e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.662156:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5557e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.662163:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760744.662168:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5557e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.662178:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5557e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.662200:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5557e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.662209:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5557e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21336064->21340159] (req 21336064->21340159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760744.662436:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508348032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38590us (38851us total) trans 4294972529 rc 0/0 00000100:00100000:3.0:1789760744.662447:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5241 00000100:00100000:3.0:1789760744.662471:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723690619577 time 723728539068 state 0x2 ref 0 00000100:00100000:3.0:1789760744.665572:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760744.666121:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508348160 00000100:00100000:2.0:1789760744.666186:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508348160 00000100:00100000:2.0:1789760744.666245:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723690619577 time 723728539068 state 0x0 ref 1 00000100:00100000:2.0:1789760744.666263:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723728799484 time 723728539068 state 0x1 ref 1 00000100:00100000:1.0:1789760744.666349:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723728799484 time 723771056781 state 0x0 ref 1 00000100:00100000:1.0:1789760744.666365:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508348160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760744.666415:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21340160, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:1.0:1789760744.666425:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55585 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.666445:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760744.666450:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760744.666456:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55585 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.666470:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55585 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760744.693032:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21340160 -> 21344256 00010000:00010000:2.0:1789760744.693064:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55585 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.693077:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55585 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.693087:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55585 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.693095:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760744.693100:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55585 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.693109:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55585 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.693139:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55585 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.693148:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55585 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21340160->21344255] (req 21340160->21344255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.693724:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508348160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26878us (27678us total) trans 4294972530 rc 0/0 00000100:00100000:2.0:1789760744.693736:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5242 00000100:00100000:2.0:1789760744.693748:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723728799484 time 723771056781 state 0x2 ref 0 00000001:00080000:3.0:1789760744.693936:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972530 is committed 00000100:00100000:1.0:1789760744.698005:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760744.698143:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508348288 00000100:00100000:2.0:1789760744.698155:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508348288 00000100:00100000:2.0:1789760744.698210:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723728799484 time 723771056781 state 0x0 ref 1 00000100:00100000:2.0:1789760744.698227:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723771317197 time 723771056781 state 0x1 ref 1 00000100:00100000:2.0:1789760744.698249:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723771317197 time 723802959634 state 0x0 ref 1 00000100:00100000:2.0:1789760744.698261:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508348288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760744.698342:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21344256, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:2.0:1789760744.698353:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5558c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.698369:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760744.698375:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.698382:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5558c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.698409:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5558c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.736436:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972531 is committed 00002000:00010000:2.0:1789760744.736541:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21344256 -> 21348352 00010000:00010000:2.0:1789760744.736557:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5558c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.736570:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5558c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.736579:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5558c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.736586:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760744.736592:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5558c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.736601:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5558c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.736624:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5558c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.736633:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5558c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21344256->21348351] (req 21344256->21348351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.736732:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508348288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38467us (38731us total) trans 4294972531 rc 0/0 00000100:00100000:2.0:1789760744.736742:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5243 00000100:00100000:2.0:1789760744.736754:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723771317197 time 723802959634 state 0x2 ref 0 00000100:00100000:1.0:1789760744.740855:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760744.741071:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508348416 00000100:00100000:3.0:1789760744.741082:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508348416 00000100:00100000:3.0:1789760744.741141:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723771317197 time 723802959634 state 0x0 ref 1 00000100:00100000:3.0:1789760744.741158:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723803220050 time 723802959634 state 0x1 ref 1 00000100:00100000:3.0:1789760744.741222:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723803220050 time 723845930985 state 0x0 ref 1 00000100:00100000:3.0:1789760744.741235:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508348416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760744.741375:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21348352, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:3.0:1789760744.741386:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55593 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.741415:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760744.741420:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760744.741429:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55593 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.741445:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55593 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760744.781173:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972532 is committed 00002000:00010000:2.0:1789760744.781708:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21348352 -> 21352448 00010000:00010000:2.0:1789760744.781726:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55593 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.781737:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55593 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.781746:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55593 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.781753:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760744.781757:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55593 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.781766:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55593 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.781789:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55593 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.781803:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55593 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21348352->21352447] (req 21348352->21352447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.781943:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508348416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40704us (41092us total) trans 4294972532 rc 0/0 00000100:00100000:2.0:1789760744.781955:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5244 00000100:00100000:2.0:1789760744.781967:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723803220050 time 723845930985 state 0x2 ref 0 00000100:00100000:0.0:1789760744.785627:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760744.785963:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508348544 00000100:00100000:2.0:1789760744.785973:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508348544 00000100:00100000:2.0:1789760744.786030:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723803220050 time 723845930985 state 0x0 ref 1 00000100:00100000:2.0:1789760744.786057:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723846191401 time 723845930985 state 0x1 ref 1 00000100:00100000:2.0:1789760744.786086:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723846191401 time 723890796744 state 0x0 ref 1 00000100:00100000:2.0:1789760744.786099:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508348544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760744.786150:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21352448, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:2.0:1789760744.786159:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5559a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.786177:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760744.786181:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.786189:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5559a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.786210:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5559a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.827237:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972533 is committed 00002000:00010000:2.0:1789760744.827397:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21352448 -> 21356544 00010000:00010000:2.0:1789760744.827423:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5559a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.827435:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5559a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.827444:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5559a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.827452:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760744.827457:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5559a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.827466:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5559a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.827564:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5559a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.827574:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5559a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21352448->21356543] (req 21352448->21356543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.827686:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508348544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41582us (42063us total) trans 4294972533 rc 0/0 00000100:00100000:2.0:1789760744.827698:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5245 00000100:00100000:2.0:1789760744.827710:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723846191401 time 723890796744 state 0x2 ref 0 00000100:00100000:1.0:1789760744.833123:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760744.833770:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508348672 00000100:00100000:2.0:1789760744.833781:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508348672 00000100:00100000:2.0:1789760744.833840:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723846191401 time 723890796744 state 0x0 ref 1 00000100:00100000:2.0:1789760744.833856:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723891057160 time 723890796744 state 0x1 ref 1 00000100:00100000:2.0:1789760744.833922:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723891057160 time 723938631799 state 0x0 ref 1 00000100:00100000:2.0:1789760744.833937:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508348672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760744.833996:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21356544, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:2.0:1789760744.834005:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.834023:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760744.834028:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.834035:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.834058:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.865021:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972534 is committed 00002000:00010000:1.0:1789760744.865111:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21356544 -> 21360640 00010000:00010000:1.0:1789760744.865127:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.865140:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.865149:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.865156:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760744.865161:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.865175:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.865197:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760744.865205:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21356544->21360639] (req 21356544->21360639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760744.865664:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508348672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31723us (32546us total) trans 4294972534 rc 0/0 00000100:00100000:1.0:1789760744.865676:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5246 00000100:00100000:1.0:1789760744.865687:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723891057160 time 723938631799 state 0x2 ref 0 00000100:00100000:3.0:1789760744.869254:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760744.870155:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508348800 00000100:00100000:2.0:1789760744.870166:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508348800 00000100:00100000:2.0:1789760744.870224:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723891057160 time 723938631799 state 0x0 ref 1 00000100:00100000:2.0:1789760744.870471:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723938892215 time 723938631799 state 0x1 ref 1 00000100:00100000:2.0:1789760744.870598:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723938892215 time 723975307603 state 0x0 ref 1 00000100:00100000:2.0:1789760744.870613:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508348800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760744.870663:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21360640, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:2.0:1789760744.870673:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.870689:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760744.870694:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.870702:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.870716:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.904818:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972535 is committed 00002000:00010000:2.0:1789760744.904920:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21360640 -> 21364736 00010000:00010000:2.0:1789760744.904936:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.904955:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.904976:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.904984:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760744.904990:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.905000:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.905025:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.905155:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21360640->21364735] (req 21360640->21364735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.905262:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508348800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34646us (36023us total) trans 4294972535 rc 0/0 00000100:00100000:2.0:1789760744.905273:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5247 00000100:00100000:2.0:1789760744.905320:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723938892215 time 723975307603 state 0x2 ref 0 00000100:00100000:0.0:1789760744.909732:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760744.909800:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508348928 00000100:00100000:2.0:1789760744.909812:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508348928 00000100:00100000:2.0:1789760744.909870:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723938892215 time 723975307603 state 0x0 ref 1 00000100:00100000:2.0:1789760744.909888:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723975568019 time 723975307603 state 0x1 ref 1 00000100:00100000:2.0:1789760744.909919:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723975568019 time 724014628971 state 0x0 ref 1 00000100:00100000:2.0:1789760744.909933:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508348928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760744.909995:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21364736, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:2.0:1789760744.910005:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.910023:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760744.910028:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.910035:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.910063:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760744.970796:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972536 is committed 00002000:00010000:2.0:1789760744.970905:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21364736 -> 21368832 00010000:00010000:2.0:1789760744.970924:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.970936:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.970951:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.970958:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760744.970964:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.970974:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.970996:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.971003:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21364736->21368831] (req 21364736->21368831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760744.971115:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508348928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61180us (61388us total) trans 4294972536 rc 0/0 00000100:00100000:2.0:1789760744.971125:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5248 00000100:00100000:2.0:1789760744.971136:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723975568019 time 724014628971 state 0x2 ref 0 00000100:00100000:1.0:1789760744.976462:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760744.976679:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508349056 00000100:00100000:2.0:1789760744.976691:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508349056 00000100:00100000:2.0:1789760744.976753:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 723975568019 time 724014628971 state 0x0 ref 1 00000100:00100000:2.0:1789760744.976770:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724014889387 time 724014628971 state 0x1 ref 1 00000100:00100000:2.0:1789760744.976833:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724014889387 time 724081542226 state 0x0 ref 1 00000100:00100000:2.0:1789760744.976845:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508349056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760744.976897:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21368832, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:2.0:1789760744.976908:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.976926:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760744.976930:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760744.976938:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760744.976954:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.080183:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972537 is committed 00002000:00010000:0.0:1789760745.082563:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21368832 -> 21372928 00010000:00010000:0.0:1789760745.082584:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.082599:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.082608:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.082615:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760745.082620:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.082630:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.082655:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.082664:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21368832->21372927] (req 21368832->21372927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.082772:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508349056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 105924us (106314us total) trans 4294972537 rc 0/0 00000100:00100000:0.0:1789760745.082783:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5249 00000100:00100000:0.0:1789760745.082794:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724014889387 time 724081542226 state 0x2 ref 0 00000100:00100000:0.0:1789760745.095139:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760745.095200:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508349184 00000100:00100000:2.0:1789760745.095210:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508349184 00000100:00100000:2.0:1789760745.095318:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724014889387 time 724081542226 state 0x0 ref 1 00000100:00100000:2.0:1789760745.095338:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724081802642 time 724081542226 state 0x1 ref 1 00000100:00100000:2.0:1789760745.095365:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724081802642 time 724200075315 state 0x0 ref 1 00000100:00100000:2.0:1789760745.095378:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508349184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760745.095433:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21372928, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:2.0:1789760745.095514:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.095532:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760745.095537:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.095544:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.095561:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760745.095988:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760745.202635:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972538 is committed 00002000:00010000:2.0:1789760745.202776:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21372928 -> 21377024 00010000:00010000:2.0:1789760745.202792:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.202805:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.202813:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.202820:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760745.202826:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.202835:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.202859:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.202867:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21372928->21377023] (req 21372928->21377023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.202964:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508349184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 107583us (107830us total) trans 4294972538 rc 0/0 00000100:00100000:2.0:1789760745.202974:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5250 00000100:00100000:2.0:1789760745.202985:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724081802642 time 724200075315 state 0x2 ref 0 00000100:00100000:3.0:1789760745.210322:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760745.211073:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508349312 00000100:00100000:2.0:1789760745.211085:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508349312 00000100:00100000:2.0:1789760745.211146:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724081802642 time 724200075315 state 0x0 ref 1 00000100:00100000:2.0:1789760745.211163:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724200335731 time 724200075315 state 0x1 ref 1 00000100:00100000:2.0:1789760745.211188:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724200335731 time 724315897881 state 0x0 ref 1 00000100:00100000:2.0:1789760745.211200:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508349312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760745.211251:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21377024, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:2.0:1789760745.211260:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.211277:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760745.211283:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.211322:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.211338:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.214651:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760745.266380:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972539 is committed 00002000:00010000:2.0:1789760745.266678:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21377024 -> 21381120 00010000:00010000:2.0:1789760745.266696:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.266709:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.266719:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.266726:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760745.266732:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.266742:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.266765:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.266777:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21377024->21381119] (req 21377024->21381119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.266874:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508349312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55671us (56595us total) trans 4294972539 rc 0/0 00000100:00100000:2.0:1789760745.266884:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5251 00000100:00100000:2.0:1789760745.266894:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724200335731 time 724315897881 state 0x2 ref 0 00000100:00100000:1.0:1789760745.274480:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760745.274710:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508349440 00000100:00100000:0.0:1789760745.274721:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508349440 00000100:00100000:0.0:1789760745.274776:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724200335731 time 724315897881 state 0x0 ref 1 00000100:00100000:0.0:1789760745.274792:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724316158297 time 724315897881 state 0x1 ref 1 00000100:00100000:0.0:1789760745.274825:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724316158297 time 724379535399 state 0x0 ref 1 00000100:00100000:0.0:1789760745.274838:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508349440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760745.274888:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21381120, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:0.0:1789760745.274899:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe555cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.274918:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760745.274922:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.274929:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe555cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.274944:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe555cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.327098:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972540 is committed 00002000:00010000:2.0:1789760745.327565:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21381120 -> 21385216 00010000:00010000:2.0:1789760745.327583:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe555cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.327598:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe555cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.327689:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe555cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.327697:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760745.327702:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe555cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.327712:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe555cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.327740:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe555cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.327747:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe555cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21381120->21385215] (req 21381120->21385215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.327839:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508349440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52998us (53365us total) trans 4294972540 rc 0/0 00000100:00100000:2.0:1789760745.327848:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5252 00000100:00100000:2.0:1789760745.327858:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724316158297 time 724379535399 state 0x2 ref 0 00000100:00100000:1.0:1789760745.344410:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760745.344542:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508349568 00000100:00100000:2.0:1789760745.344552:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508349568 00000100:00100000:2.0:1789760745.344886:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724316158297 time 724379535399 state 0x0 ref 1 00000100:00100000:2.0:1789760745.344906:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724379795815 time 724379535399 state 0x1 ref 1 00000100:00100000:2.0:1789760745.344939:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724379795815 time 724449649837 state 0x0 ref 1 00000100:00100000:2.0:1789760745.344952:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508349568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760745.345000:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21385216, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:2.0:1789760745.345010:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.345028:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760745.345033:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.345040:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.345066:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760745.403714:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972541 is committed 00002000:00010000:2.0:1789760745.404106:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21385216 -> 21389312 00010000:00010000:2.0:1789760745.404124:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.404137:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.404146:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.404153:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760745.404158:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.404167:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.404191:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.404202:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21385216->21389311] (req 21385216->21389311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.404336:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508349568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59380us (59930us total) trans 4294972541 rc 0/0 00000100:00100000:2.0:1789760745.404346:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5253 00000100:00100000:2.0:1789760745.404357:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724379795815 time 724449649837 state 0x2 ref 0 00000100:00100000:3.0:1789760745.407566:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760745.407773:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508349696 00000100:00100000:2.0:1789760745.407785:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508349696 00000100:00100000:2.0:1789760745.407842:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724379795815 time 724449649837 state 0x0 ref 1 00000100:00100000:2.0:1789760745.407858:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724449910253 time 724449649837 state 0x1 ref 1 00000100:00100000:2.0:1789760745.407885:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724449910253 time 724512594950 state 0x0 ref 1 00000100:00100000:2.0:1789760745.407897:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508349696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760745.407948:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21389312, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:2.0:1789760745.407957:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.408099:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760745.408105:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.408112:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.408129:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760745.458505:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21389312 -> 21393408 00010000:00010000:2.0:1789760745.458528:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.458543:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.458553:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.458560:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760745.458566:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.458576:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.458602:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.458610:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe555d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21389312->21393407] (req 21389312->21393407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.458757:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508349696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50856us (51194us total) trans 4294972542 rc 0/0 00000100:00100000:2.0:1789760745.458772:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5254 00000100:00100000:2.0:1789760745.458784:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724449910253 time 724512594950 state 0x2 ref 0 00000001:00080000:3.0:1789760745.458843:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972542 is committed 00000100:00100000:1.0:1789760745.462650:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760745.462743:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508349824 00000100:00100000:2.0:1789760745.462753:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508349824 00000100:00100000:2.0:1789760745.462812:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724449910253 time 724512594950 state 0x0 ref 1 00000100:00100000:2.0:1789760745.462829:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724512855366 time 724512594950 state 0x1 ref 1 00000100:00100000:2.0:1789760745.462901:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724512855366 time 724567610208 state 0x0 ref 1 00000100:00100000:2.0:1789760745.462914:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508349824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760745.463040:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21393408, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:2.0:1789760745.463061:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.463079:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760745.463084:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.463091:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.463107:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760745.488889:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972543 is committed 00002000:00010000:3.0:1789760745.489008:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21393408 -> 21397504 00010000:00010000:3.0:1789760745.489026:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.489040:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.489063:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.489071:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760745.489076:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.489091:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.489117:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.489125:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe555e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21393408->21397503] (req 21393408->21397503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.489221:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508349824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26303us (26576us total) trans 4294972543 rc 0/0 00000100:00100000:3.0:1789760745.489233:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5255 00000100:00100000:3.0:1789760745.489243:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724512855366 time 724567610208 state 0x2 ref 0 00000100:00100000:0.0:1789760745.499448:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760745.499678:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508349952 00000100:00100000:2.0:1789760745.499689:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508349952 00000100:00100000:2.0:1789760745.499745:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724512855366 time 724567610208 state 0x0 ref 1 00000100:00100000:2.0:1789760745.499760:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724567870624 time 724567610208 state 0x1 ref 1 00000100:00100000:2.0:1789760745.499824:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724567870624 time 724604534604 state 0x0 ref 1 00000100:00100000:2.0:1789760745.499837:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508349952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760745.499884:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21397504, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0xa807 00010000:00010000:2.0:1789760745.499893:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.499909:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760745.499914:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760745.499920:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.499934:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.542207:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760745.542244:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760745.542271:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760745.542307:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760745.542327:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760745.542351:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508350208 00000100:00100000:2.0:1789760745.542360:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508350208 00000100:00100000:2.0:1789760745.542419:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508350464 00000100:00100000:2.0:1789760745.542421:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508350464 00000100:00100000:2.0:1789760745.542437:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508350336 00000100:00100000:2.0:1789760745.542439:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508350336 00000100:00100000:2.0:1789760745.542449:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508350592 00000100:00100000:2.0:1789760745.542450:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508350592 00000100:00100000:2.0:1789760745.542464:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 899 00000100:00100000:2.0:1789760745.542472:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c32ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2407:x1876699508350208:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760745.542535:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c32ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2407:x1876699508350208:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 63us (339us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760745.542542:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 899 00000100:00100000:2.0:1789760745.542556:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 900 00000100:00100000:2.0:1789760745.542561:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c32f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508350464:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760745.542580:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c32f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508350464:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 20us (336us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760745.542587:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 900 00000100:00100000:2.0:1789760745.542597:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 901 00000100:00100000:2.0:1789760745.542603:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508350336:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760745.542625:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508350336:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 22us (316us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760745.542631:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 901 00000100:00100000:2.0:1789760745.542642:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 902 00000100:00100000:2.0:1789760745.542647:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508350592:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760745.542668:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508350592:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 20us (340us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760745.542676:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 902 00000100:00100000:2.0:1789760745.542692:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508350080 00000100:00100000:2.0:1789760745.542695:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508350080 00000100:00100000:2.0:1789760745.542718:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 473 00000100:00100000:2.0:1789760745.542723:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988caf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2407:x1876699508350080:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760745.542746:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988caf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2407:x1876699508350080:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 23us (474us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760745.542752:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 473 00000100:00080000:2.0:1789760745.542781:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760745.542785:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:2.0:1789760745.548963:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972544 is committed 00002000:00010000:3.0:1789760745.549113:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21397504 -> 21401600 00002000:00010000:3.0:1789760745.549118:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760745 -> 1789760746 00002000:00010000:3.0:1789760745.549120:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760745 -> 1789760746 00010000:00010000:3.0:1789760745.549136:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.549149:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.549158:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.549170:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760745.549176:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.549184:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.549206:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.549216:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe555e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21397504->21401599] (req 21397504->21401599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.549342:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508349952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49501us (49897us total) trans 4294972544 rc 0/0 00000100:00100000:3.0:1789760745.549353:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5256 00000100:00100000:3.0:1789760745.549364:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724567870624 time 724604534604 state 0x2 ref 0 00000100:00100000:2.0:1789760745.557247:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760745.557379:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508350720 00000100:00100000:0.0:1789760745.557442:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508350720 00000100:00100000:0.0:1789760745.557505:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724567870624 time 724604534604 state 0x0 ref 1 00000100:00100000:0.0:1789760745.557523:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724604795020 time 724604534604 state 0x1 ref 1 00000100:00100000:0.0:1789760745.557555:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724604795020 time 724662265735 state 0x0 ref 1 00000100:00100000:0.0:1789760745.557568:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508350720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760745.557620:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21401600, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:0.0:1789760745.557630:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe555ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.557647:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760745.557651:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.557658:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe555ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.557673:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe555ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760745.590067:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972545 is committed 00002000:00010000:1.0:1789760745.590222:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21401600 -> 21405696 00010000:00010000:1.0:1789760745.590239:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe555ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.590253:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe555ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.590268:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe555ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.590277:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760745.590282:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe555ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.590319:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe555ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.590343:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe555ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.590352:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe555ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21401600->21405695] (req 21401600->21405695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.590614:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508350720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33043us (33373us total) trans 4294972545 rc 0/0 00000100:00100000:1.0:1789760745.590626:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5257 00000100:00100000:1.0:1789760745.590636:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724604795020 time 724662265735 state 0x2 ref 0 00000100:00100000:0.0:1789760745.594545:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760745.595070:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508350848 00000100:00100000:3.0:1789760745.595082:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508350848 00000100:00100000:3.0:1789760745.595148:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724604795020 time 724662265735 state 0x0 ref 1 00000100:00100000:3.0:1789760745.595166:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724662526151 time 724662265735 state 0x1 ref 1 00000100:00100000:3.0:1789760745.595202:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724662526151 time 724699912553 state 0x0 ref 1 00000100:00100000:3.0:1789760745.595217:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508350848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760745.595269:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21405696, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:3.0:1789760745.595279:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.595333:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760745.595338:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.595345:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.595361:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.638352:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21405696 -> 21409792 00010000:00010000:3.0:1789760745.638372:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.638386:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.638407:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.638414:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760745.638419:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.638430:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.638453:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.638461:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21405696->21409791] (req 21405696->21409791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.638550:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508350848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43331us (44011us total) trans 4294972546 rc 0/0 00000100:00100000:3.0:1789760745.638560:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5258 00000100:00100000:3.0:1789760745.638570:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724662526151 time 724699912553 state 0x2 ref 0 00000001:00080000:1.0:1789760745.639232:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972546 is committed 00000100:00100000:2.0:1789760745.645594:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760745.645795:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508350976 00000100:00100000:3.0:1789760745.645806:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508350976 00000100:00100000:3.0:1789760745.645866:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724662526151 time 724699912553 state 0x0 ref 1 00000100:00100000:3.0:1789760745.645883:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724700172969 time 724699912553 state 0x1 ref 1 00000100:00100000:3.0:1789760745.645910:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724700172969 time 724750620756 state 0x0 ref 1 00000100:00100000:3.0:1789760745.645923:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508350976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760745.645979:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21409792, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:3.0:1789760745.645989:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.646007:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760745.646012:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.646019:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.646036:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760745.699338:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972547 is committed 00002000:00010000:2.0:1789760745.699552:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21409792 -> 21413888 00010000:00010000:2.0:1789760745.699568:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.699580:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.699589:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.699595:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760745.699609:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.699623:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.699645:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760745.699653:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe555fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21409792->21413887] (req 21409792->21413887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760745.699754:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508350976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53827us (54165us total) trans 4294972547 rc 0/0 00000100:00100000:2.0:1789760745.699774:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5259 00000100:00100000:2.0:1789760745.699785:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724700172969 time 724750620756 state 0x2 ref 0 00000100:00100000:2.0:1789760745.717385:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760745.717483:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508351104 00000100:00100000:1.0:1789760745.717494:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508351104 00000100:00100000:1.0:1789760745.717553:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724700172969 time 724750620756 state 0x0 ref 1 00000100:00100000:1.0:1789760745.717570:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724750881172 time 724750620756 state 0x1 ref 1 00000100:00100000:1.0:1789760745.717609:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724750881172 time 724822311442 state 0x0 ref 1 00000100:00100000:1.0:1789760745.717621:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508351104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760745.717671:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21413888, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:1.0:1789760745.717681:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55603 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.717699:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760745.717703:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.717710:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55603 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.717726:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55603 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760745.756171:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21413888 -> 21417984 00010000:00010000:1.0:1789760745.756193:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55603 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.756207:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55603 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.756216:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55603 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.756224:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760745.756231:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55603 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.756241:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55603 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.756264:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55603 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.756272:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55603 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21413888->21417983] (req 21413888->21417983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760745.756457:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508351104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38831us (39076us total) trans 4294972548 rc 0/0 00000100:00100000:1.0:1789760745.756469:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5260 00000100:00100000:1.0:1789760745.756480:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724750881172 time 724822311442 state 0x2 ref 0 00000001:00080000:1.0:1789760745.756522:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972548 is committed 00000100:00100000:1.0:1789760745.765491:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760745.765633:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508351232 00000100:00100000:3.0:1789760745.765644:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508351232 00000100:00100000:3.0:1789760745.765703:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724750881172 time 724822311442 state 0x0 ref 1 00000100:00100000:3.0:1789760745.765720:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724822571858 time 724822311442 state 0x1 ref 1 00000100:00100000:3.0:1789760745.765752:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724822571858 time 724870462084 state 0x0 ref 1 00000100:00100000:3.0:1789760745.765765:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508351232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760745.765883:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21417984, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:3.0:1789760745.765899:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5560a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.765921:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760745.765926:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.765933:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5560a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.766092:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5560a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.848177:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21417984 -> 21422080 00010000:00010000:3.0:1789760745.848197:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5560a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.848211:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5560a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.848219:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5560a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.848227:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760745.848232:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5560a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.848241:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5560a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.848262:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5560a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.848270:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5560a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21417984->21422079] (req 21417984->21422079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.848392:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508351232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 82623us (82912us total) trans 4294972549 rc 0/0 00000100:00100000:3.0:1789760745.848403:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5261 00000100:00100000:3.0:1789760745.848413:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724822571858 time 724870462084 state 0x2 ref 0 00000001:00080000:0.0:1789760745.848484:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972549 is committed 00000100:00100000:2.0:1789760745.857502:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760745.857868:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508351360 00000100:00100000:0.0:1789760745.857879:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508351360 00000100:00100000:0.0:1789760745.857938:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724822571858 time 724870462084 state 0x0 ref 1 00000100:00100000:0.0:1789760745.857954:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724870722500 time 724870462084 state 0x1 ref 1 00000100:00100000:0.0:1789760745.857992:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724870722500 time 724962694365 state 0x0 ref 1 00000100:00100000:0.0:1789760745.858006:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508351360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760745.858069:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21422080, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:0.0:1789760745.858079:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55611 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.858096:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760745.858100:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760745.858108:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55611 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.858123:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55611 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760745.975956:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21422080 -> 21426176 00010000:00010000:0.0:1789760745.975976:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55611 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.975990:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55611 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.976000:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55611 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.976008:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760745.976013:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55611 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.976024:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55611 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.976065:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55611 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760745.976075:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55611 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21422080->21426175] (req 21422080->21426175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760745.976177:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508351360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 118167us (118678us total) trans 4294972550 rc 0/0 00000100:00100000:0.0:1789760745.976188:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5262 00000100:00100000:0.0:1789760745.976201:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724870722500 time 724962694365 state 0x2 ref 0 00000001:00080000:3.0:1789760745.976587:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972550 is committed 00000100:00100000:2.0:1789760745.980347:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760745.980644:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508351488 00000100:00100000:1.0:1789760745.980655:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508351488 00000100:00100000:1.0:1789760745.980715:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724870722500 time 724962694365 state 0x0 ref 1 00000100:00100000:1.0:1789760745.980731:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724962954781 time 724962694365 state 0x1 ref 1 00000100:00100000:1.0:1789760745.980774:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724962954781 time 725085476372 state 0x0 ref 1 00000100:00100000:1.0:1789760745.980787:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508351488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760745.980837:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21426176, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:1.0:1789760745.980846:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55618 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.980863:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760745.980868:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760745.980875:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55618 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760745.980892:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55618 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.021226:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972551 is committed 00002000:00010000:3.0:1789760746.021275:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21426176 -> 21430272 00010000:00010000:3.0:1789760746.021321:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55618 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.021572:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55618 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.021658:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55618 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.021665:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760746.021670:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55618 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.021678:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55618 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.021702:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55618 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.021714:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55618 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21426176->21430271] (req 21426176->21430271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.021940:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508351488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41149us (41596us total) trans 4294972551 rc 0/0 00000100:00100000:3.0:1789760746.021952:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5263 00000100:00100000:3.0:1789760746.021963:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724962954781 time 725085476372 state 0x2 ref 0 00000100:00100000:2.0:1789760746.032649:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760746.032864:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508351616 00000100:00100000:3.0:1789760746.032874:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508351616 00000100:00100000:3.0:1789760746.032935:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 724962954781 time 725085476372 state 0x0 ref 1 00000100:00100000:3.0:1789760746.032951:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725085736788 time 725085476372 state 0x1 ref 1 00000100:00100000:3.0:1789760746.032981:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725085736788 time 725137690895 state 0x0 ref 1 00000100:00100000:3.0:1789760746.032995:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508351616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760746.033057:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21430272, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:3.0:1789760746.033068:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5561f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.033086:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.033091:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.033098:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5561f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.033120:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5561f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.149570:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972552 is committed 00002000:00010000:3.0:1789760746.150112:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21430272 -> 21434368 00010000:00010000:3.0:1789760746.150139:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5561f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.150152:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5561f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.150159:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5561f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.150166:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760746.150170:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5561f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.150178:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5561f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.150201:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5561f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.150208:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5561f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21430272->21434367] (req 21430272->21434367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.150353:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508351616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 117354us (117708us total) trans 4294972552 rc 0/0 00000100:00100000:3.0:1789760746.150364:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5264 00000100:00100000:3.0:1789760746.150374:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725085736788 time 725137690895 state 0x2 ref 0 00000100:00100000:2.0:1789760746.153481:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760746.153601:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508351744 00000100:00100000:0.0:1789760746.153611:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508351744 00000100:00100000:0.0:1789760746.153665:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725085736788 time 725137690895 state 0x0 ref 1 00000100:00100000:0.0:1789760746.153691:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725137951311 time 725137690895 state 0x1 ref 1 00000100:00100000:0.0:1789760746.153757:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725137951311 time 725258467705 state 0x0 ref 1 00000100:00100000:0.0:1789760746.153770:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508351744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760746.153819:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21434368, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:0.0:1789760746.153828:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55626 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.153845:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760746.153849:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.153856:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55626 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.153871:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55626 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.238515:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760746.238887:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972553 is committed 00002000:00010000:1.0:1789760746.239078:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21434368 -> 21438464 00010000:00010000:1.0:1789760746.239094:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55626 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.239108:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55626 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.239118:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55626 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.239132:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760746.239137:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55626 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.239146:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55626 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.239167:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55626 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.239176:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55626 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21434368->21438463] (req 21434368->21438463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.239271:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508351744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 85496us (85793us total) trans 4294972553 rc 0/0 00000100:00100000:1.0:1789760746.239282:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5265 00000100:00100000:1.0:1789760746.239328:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725137951311 time 725258467705 state 0x2 ref 0 00000100:00100000:2.0:1789760746.249591:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760746.249674:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508351872 00000100:00100000:3.0:1789760746.249695:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508351872 00000100:00100000:3.0:1789760746.249752:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725137951311 time 725258467705 state 0x0 ref 1 00000100:00100000:3.0:1789760746.249768:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725258728121 time 725258467705 state 0x1 ref 1 00000100:00100000:3.0:1789760746.249832:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725258728121 time 725354541900 state 0x0 ref 1 00000100:00100000:3.0:1789760746.249846:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508351872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760746.249894:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21438464, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:3.0:1789760746.249904:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5562d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.249923:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.249927:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.249934:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5562d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.249951:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5562d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.293622:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972554 is committed 00002000:00010000:2.0:1789760746.293901:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21438464 -> 21442560 00010000:00010000:2.0:1789760746.293916:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5562d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.293936:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5562d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.293946:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5562d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.293954:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760746.293959:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5562d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.293969:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5562d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.293997:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5562d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.294005:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5562d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21438464->21442559] (req 21438464->21442559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.294113:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508351872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44264us (44527us total) trans 4294972554 rc 0/0 00000100:00100000:2.0:1789760746.294123:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5266 00000100:00100000:2.0:1789760746.294135:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725258728121 time 725354541900 state 0x2 ref 0 00000100:00100000:2.0:1789760746.300266:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760746.300422:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508352000 00000100:00100000:3.0:1789760746.300433:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508352000 00000100:00100000:3.0:1789760746.300496:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725258728121 time 725354541900 state 0x0 ref 1 00000100:00100000:3.0:1789760746.300512:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725354802316 time 725354541900 state 0x1 ref 1 00000100:00100000:3.0:1789760746.300544:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725354802316 time 725405254825 state 0x0 ref 1 00000100:00100000:3.0:1789760746.300560:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508352000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760746.300609:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21442560, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:3.0:1789760746.300618:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55634 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.300634:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.300639:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.300646:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55634 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.300661:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55634 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.354166:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972555 is committed 00002000:00010000:0.0:1789760746.358982:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21442560 -> 21446656 00010000:00010000:0.0:1789760746.359007:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55634 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.359022:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55634 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.361199:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55634 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.361225:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760746.361231:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55634 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.361248:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55634 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.361276:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55634 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.361299:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55634 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21442560->21446655] (req 21442560->21446655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.361543:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508352000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60979us (61280us total) trans 4294972555 rc 0/0 00000100:00100000:0.0:1789760746.361555:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5267 00000100:00100000:0.0:1789760746.361566:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725354802316 time 725405254825 state 0x2 ref 0 00000100:00100000:2.0:1789760746.366773:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049891fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534070912:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760746.366845:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760746.366946:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534070912 00000100:00100000:2.0:1789760746.366955:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534070912 00000100:00100000:2.0:1789760746.367007:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:2.0:1789760746.367018:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534070912:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760746.367063:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760746.367111:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534070912:12345-0@lo:13:osp-pre-1-0.0 Request processed in 94us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.367119:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00080000:2.0:1789760746.367140:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760746.367176:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049891fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534070912:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760746.367896:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049be26e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534071040:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760746.367931:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760746.367956:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049be26f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534071552:0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.368000:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760746.368017:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534072064:0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.368049:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760746.368065:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534072576:0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.368084:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760746.368101:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:3.0:1789760746.368106:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049891fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534071168:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760746.368124:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760746.368137:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049be26dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3474:1876699534071680:0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.368155:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760746.368167:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534072192:0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.368185:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760746.368201:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534072704:0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.368220:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760746.368311:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760746.368336:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534071552 00000100:00100000:3.0:1789760746.368342:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534071552 00000100:00100000:3.0:1789760746.368386:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:3.0:1789760746.368394:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534071552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.368441:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534071552:12345-0@lo:400:kworker.0 Request processed in 47us (440us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760746.368449:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:3.0:1789760746.368469:0:8103:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534072192 00000100:00100000:3.0:1789760746.368472:0:8103:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534072192 00000100:00100000:3.0:1789760746.368497:0:8103:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534072704 00000100:00100000:3.0:1789760746.368501:0:8103:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534072704 00000100:00100000:3.0:1789760746.368521:0:8103:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:3.0:1789760746.368526:0:8103:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534072192:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.368784:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049be26fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3472:1876699534071296:0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.368822:0:8103:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534072192:12345-0@lo:400:kworker.0 Request processed in 294us (635us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760746.368829:0:8103:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00100000:3.0:1789760746.368844:0:8103:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:3.0:1789760746.368851:0:8103:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534072704:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760746.368997:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534072064 00000100:00100000:1.0:1789760746.369002:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534072064 00000100:00100000:1.0:1789760746.369030:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534072576 00000100:00100000:1.0:1789760746.369033:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534072576 00000100:00100000:1.0:1789760746.369063:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534071680 00000100:00100000:1.0:1789760746.369066:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534071680 00000100:00100000:1.0:1789760746.369084:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 903 00000100:00100000:1.0:1789760746.369091:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534072064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.369820:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760746.369861:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049be26c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534071808:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.369883:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760746.369901:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534072320:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.369917:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760746.369936:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048840c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534072832:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760746.369956:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760746.369972:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1789760746.369978:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049be26f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3472:1876699534071424:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.370000:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760746.370011:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3472:1876699534071936:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.370029:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760746.370050:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3472:1876699534072448:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.370071:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760746.370113:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534071424 00000100:00100000:2.0:1789760746.370119:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534071424 00000100:00100000:2.0:1789760746.370143:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534071936 00000100:00100000:2.0:1789760746.370145:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534071936 00000100:00100000:2.0:1789760746.370156:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534072448 00000100:00100000:2.0:1789760746.370158:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534072448 00000100:00100000:2.0:1789760746.370171:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 904 00000100:00100000:2.0:1789760746.370178:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534072576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.370216:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534072576:12345-0@lo:400:kworker.0 Request processed in 38us (2130us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.370222:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 904 00000100:00100000:2.0:1789760746.370238:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 905 00000100:00100000:2.0:1789760746.370244:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3474:x1876699534071680:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.370272:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3474:x1876699534071680:12345-0@lo:400:kworker.0 Request processed in 29us (2116us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.370278:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 905 00000100:00100000:2.0:1789760746.370312:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 906 00000100:00100000:2.0:1789760746.370318:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3472:x1876699534071424:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.370348:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3472:x1876699534071424:12345-0@lo:400:kworker.0 Request processed in 31us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.370354:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 906 00000100:00100000:2.0:1789760746.370379:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 907 00000100:00100000:2.0:1789760746.370386:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3472:x1876699534071936:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.370412:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3472:x1876699534071936:12345-0@lo:400:kworker.0 Request processed in 27us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.370417:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 907 00000100:00100000:2.0:1789760746.370427:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 908 00000100:00100000:2.0:1789760746.370432:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3472:x1876699534072448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.370457:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3472:x1876699534072448:12345-0@lo:400:kworker.0 Request processed in 25us (385us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.370463:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 908 00000100:00100000:2.0:1789760746.370480:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534071808 00000100:00100000:2.0:1789760746.370483:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534071808 00000100:00100000:2.0:1789760746.370507:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 435 00000100:00100000:2.0:1789760746.370513:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534071808:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.370542:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534071808:12345-0@lo:400:kworker.0 Request processed in 29us (657us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.370548:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 435 00000100:00100000:2.0:1789760746.370642:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534072320 00000100:00100000:2.0:1789760746.370646:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534072320 00000100:00100000:2.0:1789760746.370664:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 229 00000100:00100000:2.0:1789760746.370670:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534072320:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.370700:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534072320:12345-0@lo:400:kworker.0 Request processed in 30us (781us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.370815:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 229 00000100:00100000:2.0:1789760746.370834:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534071040 00000100:00100000:2.0:1789760746.370838:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534071040 00000100:00100000:2.0:1789760746.370853:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534071168 00000100:00100000:2.0:1789760746.370856:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534071168 00000100:00100000:2.0:1789760746.370864:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534072832 00000100:00100000:2.0:1789760746.370866:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534072832 00000100:00100000:2.0:1789760746.370877:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 373 00000100:00100000:2.0:1789760746.370882:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534071040:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760746.370896:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3836739584 avail 3813670912 00000100:00100000:2.0:1789760746.370928:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534071040:12345-0@lo:13:osp-pre-0-1.0 Request processed in 45us (2996us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.370934:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 373 00000100:00100000:2.0:1789760746.370945:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 374 00000100:00100000:2.0:1789760746.370951:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534071168:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760746.370960:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3836739584 avail 3813670912 00000100:00100000:2.0:1789760746.370983:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534071168:12345-0@lo:13:osp-pre-0-0.0 Request processed in 32us (2857us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.370989:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 374 00000100:00100000:2.0:1789760746.370998:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 375 00000100:00100000:2.0:1789760746.371003:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534072832:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789760746.371013:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760746.371034:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534072832:12345-0@lo:13:osp-pre-1-1.0 Request processed in 31us (1078us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.371049:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 375 00000100:00080000:2.0:1789760746.371066:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760746.371070:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760746.371073:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760746.371075:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760746.371125:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049be26f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3472:1876699534071424:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.371137:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3472:1876699534071936:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.371146:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3472:1876699534072448:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.371162:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048840c1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534072832:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760746.371171:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049be26c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534071808:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.371179:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534072320:0@lo:400:kworker.0 00000100:00100000:0.0:1789760746.371738:0:6460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534071296 00000100:00100000:0.0:1789760746.371745:0:6460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534071296 00000100:00100000:0.0:1789760746.371777:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:0.0:1789760746.371783:0:6460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cac050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3472:x1876699534071296:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760746.371829:0:6460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cac050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3472:x1876699534071296:12345-0@lo:400:kworker.0 Request processed in 46us (2010us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760746.371835:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00100000:2.0:1789760746.372050:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049be26fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3472:1876699534071296:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.372489:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508352128 00000100:00100000:2.0:1789760746.372494:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508352128 00000100:00100000:2.0:1789760746.372535:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725354802316 time 725405254825 state 0x0 ref 1 00000100:00100000:2.0:1789760746.372560:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725405515241 time 725405254825 state 0x1 ref 1 00000100:00100000:2.0:1789760746.372583:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725405515241 time 725477293700 state 0x0 ref 1 00000100:00100000:2.0:1789760746.372594:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508352128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760746.372638:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21446656, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:2.0:1789760746.372648:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5563b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.372665:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760746.372670:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.372677:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5563b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.372693:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5563b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.373261:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049891fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534071168:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760746.373278:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049be26dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3474:1876699534071680:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.373314:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534072192:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.373330:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049be26e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534071040:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760746.373342:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049be26f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534071552:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.373355:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534072576:0@lo:400:kworker.0 00000100:00100000:2.0:1789760746.373513:0:8103:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886361500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534072704:12345-0@lo:400:kworker.0 Request processed in 4662us (5290us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760746.373523:0:8103:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:2.0:1789760746.373638:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a3480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534072704:0@lo:400:kworker.0 00000100:00100000:1.0:1789760746.377078:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534072064:12345-0@lo:400:kworker.0 Request processed in 7977us (9018us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760746.377092:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 903 00000100:00100000:0.0:1789760746.381002:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534072064:0@lo:400:kworker.0 00002000:00010000:0.0:1789760746.433196:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21446656 -> 21450752 00010000:00010000:0.0:1789760746.433217:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5563b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.433231:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5563b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.433240:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5563b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.433247:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760746.433252:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5563b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.433262:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5563b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.433312:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5563b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.433321:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5563b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21446656->21450751] (req 21446656->21450751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.433416:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508352128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60818us (65099us total) trans 4294972556 rc 0/0 00000100:00100000:0.0:1789760746.433427:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5268 00000100:00100000:0.0:1789760746.433437:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725405515241 time 725477293700 state 0x2 ref 0 00000001:00080000:1.0:1789760746.434154:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972556 is committed 00000100:00100000:2.0:1789760746.438070:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760746.438149:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508352256 00000100:00100000:3.0:1789760746.438160:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508352256 00000100:00100000:3.0:1789760746.438219:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725405515241 time 725477293700 state 0x0 ref 1 00000100:00100000:3.0:1789760746.438236:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725477554116 time 725477293700 state 0x1 ref 1 00000100:00100000:3.0:1789760746.438267:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725477554116 time 725542977479 state 0x0 ref 1 00000100:00100000:3.0:1789760746.438280:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508352256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760746.438358:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21450752, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:3.0:1789760746.438368:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55642 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.438385:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.438390:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.438398:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55642 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.438413:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55642 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760746.487337:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21450752 -> 21454848 00010000:00010000:0.0:1789760746.487359:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55642 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.487373:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55642 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.487382:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55642 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.487399:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760746.487404:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55642 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.487414:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55642 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.487439:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55642 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.487447:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55642 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21450752->21454847] (req 21450752->21454847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.487544:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508352256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49260us (49479us total) trans 4294972557 rc 0/0 00000100:00100000:0.0:1789760746.487554:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5269 00000100:00100000:0.0:1789760746.487565:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725477554116 time 725542977479 state 0x2 ref 0 00000001:00080000:1.0:1789760746.488185:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972557 is committed 00000100:00100000:2.0:1789760746.494209:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760746.494321:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508352384 00000100:00100000:0.0:1789760746.494392:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508352384 00000100:00100000:0.0:1789760746.495142:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725477554116 time 725542977479 state 0x0 ref 1 00000100:00100000:0.0:1789760746.495160:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725543237895 time 725542977479 state 0x1 ref 1 00000100:00100000:0.0:1789760746.495313:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725543237895 time 725599997836 state 0x0 ref 1 00000100:00100000:0.0:1789760746.495710:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508352384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760746.495943:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21454848, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0xa807 00010000:00010000:1.0:1789760746.495958:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55649 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.495979:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760746.495984:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.495990:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55649 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.496006:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55649 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.526727:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972558 is committed 00002000:00010000:1.0:1789760746.526858:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21454848 -> 21458944 00002000:00010000:1.0:1789760746.526863:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760746 -> 1789760747 00002000:00010000:1.0:1789760746.526866:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760746 -> 1789760747 00010000:00010000:1.0:1789760746.526881:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55649 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.526894:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55649 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.526904:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55649 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.526912:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760746.526918:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55649 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.526927:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55649 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.526951:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55649 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.526959:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55649 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21454848->21458943] (req 21454848->21458943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.527064:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508352384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31733us (32859us total) trans 4294972558 rc 0/0 00000100:00100000:1.0:1789760746.527075:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5270 00000100:00100000:1.0:1789760746.527085:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725543237895 time 725599997836 state 0x2 ref 0 00000100:00100000:2.0:1789760746.533362:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760746.533814:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508352512 00000100:00100000:0.0:1789760746.533826:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508352512 00000100:00100000:0.0:1789760746.533894:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725543237895 time 725599997836 state 0x0 ref 1 00000100:00100000:0.0:1789760746.533911:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725600258252 time 725599997836 state 0x1 ref 1 00000100:00100000:0.0:1789760746.533943:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725600258252 time 725638653307 state 0x0 ref 1 00000100:00100000:0.0:1789760746.533956:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508352512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760746.534008:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21458944, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760746.534018:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55650 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.534203:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760746.534209:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.534217:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55650 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.534233:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55650 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760746.588652:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21458944 -> 21463040 00010000:00010000:0.0:1789760746.588674:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55650 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.588688:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55650 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.588697:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55650 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.588704:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760746.588709:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55650 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.588719:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55650 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.588741:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55650 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.588749:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55650 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21458944->21463039] (req 21458944->21463039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.588854:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508352512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54894us (55504us total) trans 4294972559 rc 0/0 00000100:00100000:0.0:1789760746.588865:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5271 00000100:00100000:0.0:1789760746.588876:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725600258252 time 725638653307 state 0x2 ref 0 00000001:00080000:1.0:1789760746.589758:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972559 is committed 00000100:00100000:0.0:1789760746.592675:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760746.593203:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508352640 00000100:00100000:3.0:1789760746.593214:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508352640 00000100:00100000:3.0:1789760746.593278:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725600258252 time 725638653307 state 0x0 ref 1 00000100:00100000:3.0:1789760746.593608:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725638913723 time 725638653307 state 0x1 ref 1 00000100:00100000:3.0:1789760746.593639:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725638913723 time 725698349508 state 0x0 ref 1 00000100:00100000:3.0:1789760746.593654:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d41450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508352640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760746.593704:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21463040, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:3.0:1789760746.593715:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55657 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.593731:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.593736:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.593744:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55657 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.593758:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55657 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760746.624768:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21463040 -> 21467136 00010000:00010000:3.0:1789760746.624790:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55657 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.624811:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55657 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.624821:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55657 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.624829:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760746.624834:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55657 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.624843:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55657 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.624866:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55657 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.624874:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55657 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21463040->21467135] (req 21463040->21467135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.624986:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d41450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508352640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31329us (32315us total) trans 4294972560 rc 0/0 00000100:00100000:3.0:1789760746.624996:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5272 00000100:00100000:3.0:1789760746.625006:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725638913723 time 725698349508 state 0x2 ref 0 00000001:00080000:1.0:1789760746.628118:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972560 is committed 00000100:00100000:3.0:1789760746.629215:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760746.629518:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508352768 00000100:00100000:1.0:1789760746.629530:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508352768 00000100:00100000:1.0:1789760746.629584:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725638913723 time 725698349508 state 0x0 ref 1 00000100:00100000:1.0:1789760746.629600:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725698609924 time 725698349508 state 0x1 ref 1 00000100:00100000:1.0:1789760746.629667:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725698609924 time 725734377148 state 0x0 ref 1 00000100:00100000:1.0:1789760746.629680:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508352768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760746.629732:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21467136, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:1.0:1789760746.629742:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5565e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.629760:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760746.629764:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760746.629771:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5565e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.629786:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5565e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760746.658550:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21467136 -> 21471232 00010000:00010000:1.0:1789760746.658573:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5565e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.658588:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5565e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.658598:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5565e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.658607:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760746.658614:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5565e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.658624:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5565e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.658651:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5565e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.658659:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5565e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21467136->21471231] (req 21467136->21471231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.658772:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508352768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29088us (29559us total) trans 4294972561 rc 0/0 00000100:00100000:1.0:1789760746.658783:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5273 00000100:00100000:1.0:1789760746.658794:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725698609924 time 725734377148 state 0x2 ref 0 00000001:00080000:0.0:1789760746.658882:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972561 is committed 00000100:00100000:0.0:1789760746.662677:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760746.662781:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508352896 00000100:00100000:2.0:1789760746.662792:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508352896 00000100:00100000:2.0:1789760746.662851:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725698609924 time 725734377148 state 0x0 ref 1 00000100:00100000:2.0:1789760746.662867:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725734637564 time 725734377148 state 0x1 ref 1 00000100:00100000:2.0:1789760746.663030:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725734637564 time 725767738897 state 0x0 ref 1 00000100:00100000:2.0:1789760746.663113:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508352896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760746.663166:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21471232, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:2.0:1789760746.663177:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55665 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.663197:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760746.663203:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.663209:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55665 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.663224:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55665 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760746.695989:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21471232 -> 21475328 00010000:00010000:2.0:1789760746.696009:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55665 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.696023:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55665 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.696052:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55665 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.696061:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760746.696066:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55665 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.696075:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55665 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.696098:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55665 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.696106:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55665 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21471232->21475327] (req 21471232->21475327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.696201:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508352896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33087us (33530us total) trans 4294972562 rc 0/0 00000100:00100000:2.0:1789760746.696213:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5274 00000100:00100000:2.0:1789760746.696225:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725734637564 time 725767738897 state 0x2 ref 0 00000001:00080000:1.0:1789760746.698860:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972562 is committed 00000100:00100000:2.0:1789760746.700141:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760746.700225:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508353024 00000100:00100000:3.0:1789760746.700235:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508353024 00000100:00100000:3.0:1789760746.700325:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725734637564 time 725767738897 state 0x0 ref 1 00000100:00100000:3.0:1789760746.700342:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725767999313 time 725767738897 state 0x1 ref 1 00000100:00100000:3.0:1789760746.700367:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725767999313 time 725805077988 state 0x0 ref 1 00000100:00100000:3.0:1789760746.700393:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508353024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760746.700446:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21475328, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:3.0:1789760746.700457:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5566c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.700475:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.700480:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.700486:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5566c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.700502:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5566c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.725892:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972563 is committed 00002000:00010000:3.0:1789760746.726737:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21475328 -> 21479424 00010000:00010000:3.0:1789760746.726754:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5566c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.726767:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5566c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.726783:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5566c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.726790:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760746.726796:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5566c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.726805:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5566c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.726828:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5566c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.726836:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5566c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21475328->21479423] (req 21475328->21479423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.727029:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508353024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26642us (26891us total) trans 4294972563 rc 0/0 00000100:00100000:3.0:1789760746.727056:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5275 00000100:00100000:3.0:1789760746.727067:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725767999313 time 725805077988 state 0x2 ref 0 00000100:00100000:1.0:1789760746.730449:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760746.730546:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508353152 00000100:00100000:2.0:1789760746.730557:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508353152 00000100:00100000:2.0:1789760746.730613:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725767999313 time 725805077988 state 0x0 ref 1 00000100:00100000:2.0:1789760746.730629:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725805338404 time 725805077988 state 0x1 ref 1 00000100:00100000:2.0:1789760746.730694:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725805338404 time 725835403216 state 0x0 ref 1 00000100:00100000:2.0:1789760746.730707:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508353152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760746.730757:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21479424, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:2.0:1789760746.730768:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55673 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.730786:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760746.730792:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.730799:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55673 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.730820:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55673 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760746.763100:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972564 is committed 00002000:00010000:3.0:1789760746.763213:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21479424 -> 21483520 00010000:00010000:3.0:1789760746.763229:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55673 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.763248:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55673 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.763256:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55673 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.763263:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760746.763267:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55673 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.763275:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55673 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.763322:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55673 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.763330:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55673 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21479424->21483519] (req 21479424->21483519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.763579:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508353152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32869us (33195us total) trans 4294972564 rc 0/0 00000100:00100000:3.0:1789760746.763590:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5276 00000100:00100000:3.0:1789760746.763600:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725805338404 time 725835403216 state 0x2 ref 0 00000100:00100000:0.0:1789760746.767935:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760746.768098:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508353280 00000100:00100000:2.0:1789760746.768110:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508353280 00000100:00100000:2.0:1789760746.768170:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725805338404 time 725835403216 state 0x0 ref 1 00000100:00100000:2.0:1789760746.768188:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725835663632 time 725835403216 state 0x1 ref 1 00000100:00100000:2.0:1789760746.768267:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725835663632 time 725872976842 state 0x0 ref 1 00000100:00100000:2.0:1789760746.768280:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508353280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760746.768641:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21483520, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:2.0:1789760746.768652:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5567a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.768669:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760746.768673:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760746.768681:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5567a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.768696:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5567a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760746.796779:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972565 is committed 00002000:00010000:2.0:1789760746.796890:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21483520 -> 21487616 00010000:00010000:2.0:1789760746.796906:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5567a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.796920:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5567a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.796929:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5567a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.796936:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760746.796941:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5567a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.797026:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5567a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.797060:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5567a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760746.797068:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5567a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21483520->21487615] (req 21483520->21487615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760746.797203:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508353280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28918us (29271us total) trans 4294972565 rc 0/0 00000100:00100000:2.0:1789760746.797214:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5277 00000100:00100000:2.0:1789760746.797226:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725835663632 time 725872976842 state 0x2 ref 0 00000100:00100000:2.0:1789760746.801120:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760746.801216:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508353408 00000100:00100000:3.0:1789760746.801226:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508353408 00000100:00100000:3.0:1789760746.801580:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725835663632 time 725872976842 state 0x0 ref 1 00000100:00100000:3.0:1789760746.801599:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725873237258 time 725872976842 state 0x1 ref 1 00000100:00100000:0.0:1789760746.801682:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725873237258 time 725906390595 state 0x0 ref 1 00000100:00100000:0.0:1789760746.801698:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508353408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760746.801751:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21487616, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760746.801761:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55681 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.801778:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760746.801783:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.801790:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55681 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.801806:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55681 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.828662:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972566 is committed 00002000:00010000:0.0:1789760746.828843:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21487616 -> 21491712 00010000:00010000:0.0:1789760746.828861:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55681 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.828875:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55681 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.828884:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55681 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.828891:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760746.828896:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55681 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.828906:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55681 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.828931:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55681 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.828940:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55681 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21487616->21491711] (req 21487616->21491711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760746.829122:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508353408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27423us (28007us total) trans 4294972566 rc 0/0 00000100:00100000:0.0:1789760746.829137:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5278 00000100:00100000:0.0:1789760746.829148:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725873237258 time 725906390595 state 0x2 ref 0 00000100:00100000:2.0:1789760746.833397:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760746.833571:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508353536 00000100:00100000:3.0:1789760746.833582:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508353536 00000100:00100000:3.0:1789760746.833641:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725873237258 time 725906390595 state 0x0 ref 1 00000100:00100000:3.0:1789760746.833658:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725906651011 time 725906390595 state 0x1 ref 1 00000100:00100000:3.0:1789760746.833691:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725906651011 time 725938401254 state 0x0 ref 1 00000100:00100000:3.0:1789760746.833706:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508353536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760746.833758:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21491712, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:3.0:1789760746.833769:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55688 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.833787:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760746.833791:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.833798:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55688 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.833812:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55688 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760746.858612:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972567 is committed 00002000:00010000:3.0:1789760746.858720:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21491712 -> 21495808 00010000:00010000:3.0:1789760746.858737:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55688 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.858751:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55688 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.858761:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55688 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.858768:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760746.858773:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55688 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.858782:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55688 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.858805:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55688 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.858813:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55688 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21491712->21495807] (req 21491712->21495807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.858911:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508353536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25203us (25528us total) trans 4294972567 rc 0/0 00000100:00100000:3.0:1789760746.858922:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5279 00000100:00100000:3.0:1789760746.858932:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725906651011 time 725938401254 state 0x2 ref 0 00000100:00100000:2.0:1789760746.862636:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760746.862719:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508353664 00000100:00100000:0.0:1789760746.862729:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508353664 00000100:00100000:0.0:1789760746.862783:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725906651011 time 725938401254 state 0x0 ref 1 00000100:00100000:0.0:1789760746.862798:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725938661670 time 725938401254 state 0x1 ref 1 00000100:00100000:0.0:1789760746.862827:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725938661670 time 725967537951 state 0x0 ref 1 00000100:00100000:0.0:1789760746.862841:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508353664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760746.862896:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21495808, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760746.862906:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5568f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.862924:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760746.862929:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.862936:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5568f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.862962:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5568f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.895727:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972568 is committed 00002000:00010000:3.0:1789760746.895839:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21495808 -> 21499904 00010000:00010000:3.0:1789760746.895854:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5568f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.895868:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5568f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.895877:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5568f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.895886:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760746.895890:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5568f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.895900:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5568f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.895922:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5568f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.895929:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5568f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21495808->21499903] (req 21495808->21499903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.896111:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508353664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33265us (33477us total) trans 4294972568 rc 0/0 00000100:00100000:3.0:1789760746.896121:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5280 00000100:00100000:3.0:1789760746.896133:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725938661670 time 725967537951 state 0x2 ref 0 00000100:00100000:2.0:1789760746.899928:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760746.900074:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508353792 00000100:00100000:0.0:1789760746.900085:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508353792 00000100:00100000:0.0:1789760746.900144:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725938661670 time 725967537951 state 0x0 ref 1 00000100:00100000:0.0:1789760746.900160:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725967798367 time 725967537951 state 0x1 ref 1 00000100:00100000:0.0:1789760746.900191:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725967798367 time 726004901486 state 0x0 ref 1 00000100:00100000:0.0:1789760746.900204:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508353792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760746.900255:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21499904, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760746.900266:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55696 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.900320:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760746.900326:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.900333:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55696 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.900352:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55696 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760746.932833:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21499904 -> 21504000 00010000:00010000:1.0:1789760746.932855:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55696 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.932870:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55696 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.932880:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55696 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.932888:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760746.932894:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55696 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.932903:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55696 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.932927:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55696 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760746.932935:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55696 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21499904->21503999] (req 21499904->21503999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760746.933112:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508353792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32905us (33188us total) trans 4294972569 rc 0/0 00000100:00100000:1.0:1789760746.933124:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5281 00000100:00100000:1.0:1789760746.933135:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725967798367 time 726004901486 state 0x2 ref 0 00000001:00080000:2.0:1789760746.934786:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972569 is committed 00000100:00100000:1.0:1789760746.937119:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760746.937196:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508353920 00000100:00100000:3.0:1789760746.937207:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508353920 00000100:00100000:3.0:1789760746.937267:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 725967798367 time 726004901486 state 0x0 ref 1 00000100:00100000:3.0:1789760746.937631:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726005161902 time 726004901486 state 0x1 ref 1 00000100:00100000:3.0:1789760746.937666:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726005161902 time 726042375925 state 0x0 ref 1 00000100:00100000:3.0:1789760746.937682:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508353920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760746.937733:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21504000, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:3.0:1789760746.937741:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5569d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.937758:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760746.937763:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.937769:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5569d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.937782:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5569d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760746.959653:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21504000 -> 21508096 00010000:00010000:3.0:1789760746.959674:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5569d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.959693:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5569d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.959703:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5569d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.959711:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760746.959716:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5569d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.959726:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5569d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.959751:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5569d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.959760:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5569d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21504000->21508095] (req 21504000->21508095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.959844:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508353920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22160us (22728us total) trans 4294972570 rc 0/0 00000100:00100000:3.0:1789760746.959854:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5282 00000100:00100000:3.0:1789760746.959864:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726005161902 time 726042375925 state 0x2 ref 0 00000001:00080000:2.0:1789760746.960053:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972570 is committed 00000100:00100000:3.0:1789760746.964103:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760746.964323:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508354048 00000100:00100000:0.0:1789760746.964335:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508354048 00000100:00100000:0.0:1789760746.964400:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726005161902 time 726042375925 state 0x0 ref 1 00000100:00100000:0.0:1789760746.964417:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726042636341 time 726042375925 state 0x1 ref 1 00000100:00100000:0.0:1789760746.964439:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726042636341 time 726069148961 state 0x0 ref 1 00000100:00100000:0.0:1789760746.964457:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508354048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760746.964507:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21508096, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760746.964518:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.964537:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760746.964542:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760746.964549:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760746.964565:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760746.993985:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972571 is committed 00002000:00010000:3.0:1789760746.994157:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21508096 -> 21512192 00010000:00010000:3.0:1789760746.994173:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.994187:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.994197:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.994205:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760746.994210:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.994220:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.994243:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.994252:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21508096->21512191] (req 21508096->21512191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.994513:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508354048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30053us (30414us total) trans 4294972571 rc 0/0 00000100:00100000:3.0:1789760746.994524:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5283 00000100:00100000:3.0:1789760746.994536:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726042636341 time 726069148961 state 0x2 ref 0 00000100:00100000:3.0:1789760747.009345:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760747.009452:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508354176 00000100:00100000:1.0:1789760747.009462:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508354176 00000100:00100000:1.0:1789760747.009537:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726042636341 time 726069148961 state 0x0 ref 1 00000100:00100000:1.0:1789760747.009554:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726069409377 time 726069148961 state 0x1 ref 1 00000100:00100000:1.0:1789760747.009592:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726069409377 time 726114302477 state 0x0 ref 1 00000100:00100000:1.0:1789760747.009606:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508354176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760747.009656:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21512192, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:1.0:1789760747.009666:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe556ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.009684:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760747.009688:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.009695:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe556ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.009711:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe556ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760747.009902:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760747.029213:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972572 is committed 00002000:00010000:1.0:1789760747.029339:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21512192 -> 21516288 00010000:00010000:1.0:1789760747.029356:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe556ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.029368:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe556ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.029377:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe556ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.029395:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760747.029401:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe556ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.029410:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe556ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.029432:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe556ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.029439:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe556ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21512192->21516287] (req 21512192->21516287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.029532:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508354176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19923us (20192us total) trans 4294972572 rc 0/0 00000100:00100000:1.0:1789760747.029543:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5284 00000100:00100000:1.0:1789760747.029554:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726069409377 time 726114302477 state 0x2 ref 0 00000100:00100000:2.0:1789760747.033780:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760747.033869:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508354304 00000100:00100000:3.0:1789760747.033879:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508354304 00000100:00100000:3.0:1789760747.033937:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726069409377 time 726114302477 state 0x0 ref 1 00000100:00100000:3.0:1789760747.033954:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726114562893 time 726114302477 state 0x1 ref 1 00000100:00100000:3.0:1789760747.033986:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726114562893 time 726138695484 state 0x0 ref 1 00000100:00100000:3.0:1789760747.033998:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508354304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760747.034058:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21516288, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:3.0:1789760747.034068:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe556b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.034086:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760747.034091:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.034100:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe556b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.034116:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe556b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.093581:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21516288 -> 21520384 00010000:00010000:0.0:1789760747.093601:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe556b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.093613:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe556b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.093621:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe556b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.093628:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.093632:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe556b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.093641:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe556b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.093664:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe556b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.093672:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe556b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21516288->21520383] (req 21516288->21520383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.093769:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508354304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59767us (59993us total) trans 4294972573 rc 0/0 00000100:00100000:0.0:1789760747.093781:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5285 00000100:00100000:0.0:1789760747.093792:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726114562893 time 726138695484 state 0x2 ref 0 00000001:00080000:3.0:1789760747.094325:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972573 is committed 00000100:00100000:3.0:1789760747.105429:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.105533:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508354432 00000100:00100000:0.0:1789760747.105543:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508354432 00000100:00100000:0.0:1789760747.105608:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726114562893 time 726138695484 state 0x0 ref 1 00000100:00100000:0.0:1789760747.105624:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726138955900 time 726138695484 state 0x1 ref 1 00000100:00100000:0.0:1789760747.105657:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726138955900 time 726210367325 state 0x0 ref 1 00000100:00100000:0.0:1789760747.105675:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508354432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.105730:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21520384, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760747.105740:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.105760:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.105765:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.105772:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.105789:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.136848:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972574 is committed 00002000:00010000:0.0:1789760747.137021:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21520384 -> 21524480 00010000:00010000:0.0:1789760747.137050:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.137065:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.137074:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.137081:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.137086:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.137096:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.137119:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.137129:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21520384->21524479] (req 21520384->21524479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.137265:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508354432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31590us (31839us total) trans 4294972574 rc 0/0 00000100:00100000:0.0:1789760747.137276:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5286 00000100:00100000:0.0:1789760747.137329:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726138955900 time 726210367325 state 0x2 ref 0 00000100:00100000:1.0:1789760747.147113:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.147319:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508354560 00000100:00100000:0.0:1789760747.147330:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508354560 00000100:00100000:0.0:1789760747.147718:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726138955900 time 726210367325 state 0x0 ref 1 00000100:00100000:0.0:1789760747.147734:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726210627741 time 726210367325 state 0x1 ref 1 00000100:00100000:0.0:1789760747.147768:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726210627741 time 726252478148 state 0x0 ref 1 00000100:00100000:0.0:1789760747.147781:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508354560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.147835:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21524480, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760747.147847:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.147865:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.147869:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.147876:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.147896:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.183949:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972575 is committed 00002000:00010000:2.0:1789760747.184302:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21524480 -> 21528576 00010000:00010000:2.0:1789760747.184319:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.184331:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.184341:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.184349:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760747.184354:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.184363:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.184447:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.184458:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe556c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21524480->21528575] (req 21524480->21528575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.184595:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508354560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36811us (37487us total) trans 4294972575 rc 0/0 00000100:00100000:2.0:1789760747.184606:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5287 00000100:00100000:2.0:1789760747.184616:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726210627741 time 726252478148 state 0x2 ref 0 00000100:00100000:2.0:1789760747.188987:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.189220:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508354688 00000100:00100000:0.0:1789760747.189232:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508354688 00000100:00100000:0.0:1789760747.189663:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726210627741 time 726252478148 state 0x0 ref 1 00000100:00100000:0.0:1789760747.189683:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726252738564 time 726252478148 state 0x1 ref 1 00000100:00100000:0.0:1789760747.189715:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726252738564 time 726294425195 state 0x0 ref 1 00000100:00100000:0.0:1789760747.189730:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508354688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.189787:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21528576, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760747.189798:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.189816:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.189821:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.189828:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.189845:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.224465:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972576 is committed 00002000:00010000:0.0:1789760747.224851:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21528576 -> 21532672 00010000:00010000:0.0:1789760747.224867:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.224882:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.224891:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.224900:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.224905:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.224915:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.224938:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.224946:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21528576->21532671] (req 21528576->21532671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.225141:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508354688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35409us (36157us total) trans 4294972576 rc 0/0 00000100:00100000:0.0:1789760747.225152:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5288 00000100:00100000:0.0:1789760747.225162:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726252738564 time 726294425195 state 0x2 ref 0 00000100:00100000:3.0:1789760747.228828:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.228916:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508354816 00000100:00100000:0.0:1789760747.228926:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508354816 00000100:00100000:0.0:1789760747.229059:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726252738564 time 726294425195 state 0x0 ref 1 00000100:00100000:0.0:1789760747.229075:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726294685611 time 726294425195 state 0x1 ref 1 00000100:00100000:0.0:1789760747.229110:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726294685611 time 726333820413 state 0x0 ref 1 00000100:00100000:0.0:1789760747.229132:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508354816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.229180:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21532672, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760747.229190:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.229209:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.229213:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.229221:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.229236:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.262447:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760747.285586:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972577 is committed 00002000:00010000:0.0:1789760747.285691:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21532672 -> 21536768 00010000:00010000:0.0:1789760747.285709:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.285723:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.285733:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.285740:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.285745:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.285756:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.285779:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.285787:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21532672->21536767] (req 21532672->21536767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.285891:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508354816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56757us (57067us total) trans 4294972577 rc 0/0 00000100:00100000:0.0:1789760747.285902:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5289 00000100:00100000:0.0:1789760747.285914:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726294685611 time 726333820413 state 0x2 ref 0 00000100:00100000:0.0:1789760747.292771:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760747.292884:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508354944 00000100:00100000:2.0:1789760747.292895:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508354944 00000100:00100000:2.0:1789760747.293516:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726294685611 time 726333820413 state 0x0 ref 1 00000100:00100000:2.0:1789760747.293534:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726334080829 time 726333820413 state 0x1 ref 1 00000100:00100000:2.0:1789760747.293615:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726334080829 time 726398324193 state 0x0 ref 1 00000100:00100000:2.0:1789760747.293630:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508354944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760747.293683:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21536768, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:2.0:1789760747.293692:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe556d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.293718:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760747.293722:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.293731:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe556d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.293748:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe556d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.334884:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972578 is committed 00002000:00010000:2.0:1789760747.335110:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21536768 -> 21540864 00010000:00010000:2.0:1789760747.335128:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe556d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.335141:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe556d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.335151:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe556d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.335158:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760747.335164:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe556d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.335174:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe556d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.335199:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe556d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.335208:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe556d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21536768->21540863] (req 21536768->21540863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760747.335341:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508354944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41708us (42575us total) trans 4294972578 rc 0/0 00000100:00100000:2.0:1789760747.335352:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5290 00000100:00100000:2.0:1789760747.335363:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726334080829 time 726398324193 state 0x2 ref 0 00000100:00100000:1.0:1789760747.338955:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.339084:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508355072 00000100:00100000:0.0:1789760747.339104:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508355072 00000100:00100000:0.0:1789760747.339604:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726334080829 time 726398324193 state 0x0 ref 1 00000100:00100000:0.0:1789760747.339625:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726398584609 time 726398324193 state 0x1 ref 1 00000100:00100000:0.0:1789760747.339645:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726398584609 time 726444356128 state 0x0 ref 1 00000100:00100000:0.0:1789760747.339660:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508355072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.339709:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21540864, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760747.339719:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.339737:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.339742:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.339749:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.339764:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.387275:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972579 is committed 00002000:00010000:0.0:1789760747.387561:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21540864 -> 21544960 00010000:00010000:0.0:1789760747.387578:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.387593:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.387603:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.387617:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.387623:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.387633:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.387658:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.387667:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21540864->21544959] (req 21540864->21544959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.387794:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508355072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48133us (48844us total) trans 4294972579 rc 0/0 00000100:00100000:0.0:1789760747.387805:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5291 00000100:00100000:0.0:1789760747.387816:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726398584609 time 726444356128 state 0x2 ref 0 00000100:00100000:0.0:1789760747.391217:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760747.391241:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048840c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534072960:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760747.391351:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760747.391571:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534072960 00000100:00100000:1.0:1789760747.391580:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534072960 00000100:00100000:1.0:1789760747.391671:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:1.0:1789760747.391681:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534072960:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760747.391724:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760747.391797:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534072960:12345-0@lo:41:osp-pre-0-1.0 Request processed in 116us (445us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760747.391808:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00100000:0.0:1789760747.392003:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048840c3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534072960:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760747.395649:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.395738:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508355200 00000100:00100000:0.0:1789760747.395749:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508355200 00000100:00100000:0.0:1789760747.395807:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726398584609 time 726444356128 state 0x0 ref 1 00000100:00100000:0.0:1789760747.395827:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726444616544 time 726444356128 state 0x1 ref 1 00000100:00100000:0.0:1789760747.395859:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726444616544 time 726500569458 state 0x0 ref 1 00000100:00100000:0.0:1789760747.395872:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508355200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.395925:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21544960, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760747.395935:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.395967:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.395972:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.395981:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.395995:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.422701:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972580 is committed 00002000:00010000:0.0:1789760747.423566:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21544960 -> 21549056 00010000:00010000:0.0:1789760747.423586:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.423598:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.423607:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.423618:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.423622:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.423632:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.423659:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.423667:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21544960->21549055] (req 21544960->21549055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.423768:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508355200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27893us (28126us total) trans 4294972580 rc 0/0 00000100:00100000:0.0:1789760747.423779:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5292 00000100:00100000:0.0:1789760747.423790:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726444616544 time 726500569458 state 0x2 ref 0 00000100:00100000:3.0:1789760747.428108:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.428205:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508355328 00000100:00100000:0.0:1789760747.428216:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508355328 00000100:00100000:0.0:1789760747.428279:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726444616544 time 726500569458 state 0x0 ref 1 00000100:00100000:0.0:1789760747.429084:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726500829874 time 726500569458 state 0x1 ref 1 00000100:00100000:0.0:1789760747.429114:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726500829874 time 726533823679 state 0x0 ref 1 00000100:00100000:0.0:1789760747.429136:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508355328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.429188:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21549056, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760747.429198:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.429217:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.429222:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.429229:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.429244:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.462396:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972581 is committed 00002000:00010000:1.0:1789760747.462581:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21549056 -> 21553152 00010000:00010000:1.0:1789760747.462597:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.462611:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.462619:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.462625:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760747.462630:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.462638:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.462660:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.462667:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe556ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21549056->21553151] (req 21549056->21553151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.462756:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508355328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33623us (34653us total) trans 4294972581 rc 0/0 00000100:00100000:1.0:1789760747.462771:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5293 00000100:00100000:1.0:1789760747.462782:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726500829874 time 726533823679 state 0x2 ref 0 00000100:00100000:1.0:1789760747.468570:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.468692:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508355456 00000100:00100000:0.0:1789760747.468703:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508355456 00000100:00100000:0.0:1789760747.468762:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726500829874 time 726533823679 state 0x0 ref 1 00000100:00100000:0.0:1789760747.468779:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726534084095 time 726533823679 state 0x1 ref 1 00000100:00100000:0.0:1789760747.468811:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726534084095 time 726573520898 state 0x0 ref 1 00000100:00100000:0.0:1789760747.468824:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508355456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.468877:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21553152, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0xa807 00010000:00010000:0.0:1789760747.468887:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.468951:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.468957:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.468964:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.468980:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.517972:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21553152 -> 21557248 00002000:00010000:0.0:1789760747.517982:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760747 -> 1789760748 00002000:00010000:0.0:1789760747.517985:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760747 -> 1789760748 00010000:00010000:0.0:1789760747.518000:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.518013:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.518023:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.518040:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.518045:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.518055:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.518087:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.518096:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe556f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21553152->21557247] (req 21553152->21557247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.518194:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508355456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49366us (49628us total) trans 4294972582 rc 0/0 00000100:00100000:0.0:1789760747.518205:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5294 00000100:00100000:0.0:1789760747.518215:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726534084095 time 726573520898 state 0x2 ref 0 00000001:00080000:2.0:1789760747.520052:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972582 is committed 00000100:00100000:3.0:1789760747.521471:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.521663:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508355584 00000100:00100000:0.0:1789760747.521674:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508355584 00000100:00100000:0.0:1789760747.521733:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726534084095 time 726573520898 state 0x0 ref 1 00000100:00100000:0.0:1789760747.521750:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726573781314 time 726573520898 state 0x1 ref 1 00000100:00100000:0.0:1789760747.521772:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726573781314 time 726626483356 state 0x0 ref 1 00000100:00100000:0.0:1789760747.521786:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508355584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.521841:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21557248, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:0.0:1789760747.521850:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe556f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.521867:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.521872:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.521879:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe556f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.521894:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe556f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.553781:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972583 is committed 00002000:00010000:0.0:1789760747.553876:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21557248 -> 21561344 00010000:00010000:0.0:1789760747.553893:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe556f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.553907:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe556f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.553917:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe556f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.553924:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.553930:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe556f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.553939:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe556f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.553963:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe556f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.553972:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe556f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21557248->21561343] (req 21557248->21561343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.554077:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508355584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32287us (32606us total) trans 4294972583 rc 0/0 00000100:00100000:0.0:1789760747.554089:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5295 00000100:00100000:0.0:1789760747.554101:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726573781314 time 726626483356 state 0x2 ref 0 00000100:00100000:3.0:1789760747.558216:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760747.558585:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508355712 00000100:00100000:1.0:1789760747.558596:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508355712 00000100:00100000:1.0:1789760747.558658:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726573781314 time 726626483356 state 0x0 ref 1 00000100:00100000:1.0:1789760747.558675:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726626743772 time 726626483356 state 0x1 ref 1 00000100:00100000:1.0:1789760747.558700:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726626743772 time 726663410994 state 0x0 ref 1 00000100:00100000:1.0:1789760747.558713:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508355712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760747.558799:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21561344, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:2.0:1789760747.558811:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe556ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.558830:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760747.558835:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.558843:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe556ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.558860:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe556ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760747.598683:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972584 is committed 00002000:00010000:0.0:1789760747.598820:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21561344 -> 21565440 00010000:00010000:0.0:1789760747.598837:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe556ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.598849:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe556ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.598858:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe556ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.598865:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.598870:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe556ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.598879:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe556ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.598902:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe556ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.598915:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe556ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21561344->21565439] (req 21561344->21565439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.599011:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508355712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40295us (40798us total) trans 4294972584 rc 0/0 00000100:00100000:0.0:1789760747.599021:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5296 00000100:00100000:0.0:1789760747.599041:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726626743772 time 726663410994 state 0x2 ref 0 00000100:00100000:1.0:1789760747.602969:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.603111:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508355840 00000100:00100000:0.0:1789760747.603122:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508355840 00000100:00100000:0.0:1789760747.603181:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726626743772 time 726663410994 state 0x0 ref 1 00000100:00100000:0.0:1789760747.603198:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726663671410 time 726663410994 state 0x1 ref 1 00000100:00100000:0.0:1789760747.603227:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726663671410 time 726707937364 state 0x0 ref 1 00000100:00100000:0.0:1789760747.603240:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508355840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.603339:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21565440, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:0.0:1789760747.603350:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55706 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.603368:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.603372:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.603379:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55706 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.603400:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55706 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.644742:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972585 is committed 00002000:00010000:0.0:1789760747.644859:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21565440 -> 21569536 00010000:00010000:0.0:1789760747.644876:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55706 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.644888:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55706 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.644896:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55706 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.644903:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.644908:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55706 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.644916:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55706 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.644938:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55706 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.644946:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55706 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21565440->21569535] (req 21565440->21569535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.645058:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508355840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41815us (42094us total) trans 4294972585 rc 0/0 00000100:00100000:0.0:1789760747.645068:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5297 00000100:00100000:0.0:1789760747.645079:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726663671410 time 726707937364 state 0x2 ref 0 00000100:00100000:3.0:1789760747.651173:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.651260:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508355968 00000100:00100000:0.0:1789760747.651271:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508355968 00000100:00100000:0.0:1789760747.651358:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726663671410 time 726707937364 state 0x0 ref 1 00000100:00100000:0.0:1789760747.651384:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726708197780 time 726707937364 state 0x1 ref 1 00000100:00100000:0.0:1789760747.651415:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726708197780 time 726756125237 state 0x0 ref 1 00000100:00100000:0.0:1789760747.651428:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508355968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.651477:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21569536, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:0.0:1789760747.651486:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5570d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.651505:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.651510:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.651517:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5570d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.651531:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5570d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760747.690783:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21569536 -> 21573632 00010000:00010000:1.0:1789760747.690805:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5570d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.690817:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5570d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.690826:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5570d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.690832:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760747.690838:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5570d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.690847:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5570d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.690869:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5570d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.690876:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5570d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21569536->21573631] (req 21569536->21573631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.690977:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508355968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39547us (39809us total) trans 4294972586 rc 0/0 00000100:00100000:1.0:1789760747.690987:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5298 00000100:00100000:1.0:1789760747.690997:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726708197780 time 726756125237 state 0x2 ref 0 00000001:00080000:3.0:1789760747.691963:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972586 is committed 00000100:00100000:3.0:1789760747.707465:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.707637:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508356096 00000100:00100000:0.0:1789760747.707647:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508356096 00000100:00100000:0.0:1789760747.707714:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726708197780 time 726756125237 state 0x0 ref 1 00000100:00100000:0.0:1789760747.707731:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726756385653 time 726756125237 state 0x1 ref 1 00000100:00100000:0.0:1789760747.707764:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726756385653 time 726812473865 state 0x0 ref 1 00000100:00100000:0.0:1789760747.707777:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508356096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.707827:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21573632, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:0.0:1789760747.707837:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55714 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.707854:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.707858:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.707865:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55714 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.707879:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55714 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.742264:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21573632 -> 21577728 00010000:00010000:0.0:1789760747.742304:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55714 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.742323:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55714 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.742332:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55714 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.742340:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.742345:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55714 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.742355:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55714 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.742452:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55714 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.742461:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55714 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21573632->21577727] (req 21573632->21577727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.742518:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972587 is committed 00000100:00100000:0.0:1789760747.742661:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508356096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34880us (35200us total) trans 4294972587 rc 0/0 00000100:00100000:0.0:1789760747.742671:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5299 00000100:00100000:0.0:1789760747.742682:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726756385653 time 726812473865 state 0x2 ref 0 00000100:00100000:0.0:1789760747.746667:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760747.746834:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508356224 00000100:00100000:2.0:1789760747.746844:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508356224 00000100:00100000:2.0:1789760747.746901:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726756385653 time 726812473865 state 0x0 ref 1 00000100:00100000:2.0:1789760747.746917:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726812734281 time 726812473865 state 0x1 ref 1 00000100:00100000:2.0:1789760747.746948:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726812734281 time 726851658100 state 0x0 ref 1 00000100:00100000:2.0:1789760747.746961:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508356224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760747.747012:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21577728, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:2.0:1789760747.747023:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5571b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.747050:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760747.747055:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.747132:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5571b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.747147:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5571b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.804788:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972588 is committed 00002000:00010000:0.0:1789760747.804893:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21577728 -> 21581824 00010000:00010000:0.0:1789760747.804909:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5571b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.804923:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5571b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.804932:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5571b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.804940:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.804945:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5571b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.804955:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5571b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.804980:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5571b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.804989:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5571b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21577728->21581823] (req 21577728->21581823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760747.805106:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508356224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58142us (58443us total) trans 4294972588 rc 0/0 00000100:00100000:0.0:1789760747.805116:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5300 00000100:00100000:0.0:1789760747.805126:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726812734281 time 726851658100 state 0x2 ref 0 00000100:00100000:3.0:1789760747.812882:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760747.812983:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508356352 00000100:00100000:1.0:1789760747.812994:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508356352 00000100:00100000:1.0:1789760747.813062:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726812734281 time 726851658100 state 0x0 ref 1 00000100:00100000:1.0:1789760747.813079:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726851918516 time 726851658100 state 0x1 ref 1 00000100:00100000:1.0:1789760747.813125:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726851918516 time 726917835340 state 0x0 ref 1 00000100:00100000:1.0:1789760747.813138:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508356352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760747.813190:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21581824, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:1.0:1789760747.813200:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55722 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.813217:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760747.813222:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.813230:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55722 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.813246:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55722 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.844070:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972589 is committed 00002000:00010000:1.0:1789760747.844663:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21581824 -> 21585920 00010000:00010000:1.0:1789760747.844679:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55722 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.844693:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55722 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.844702:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55722 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.844709:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760747.844714:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55722 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.844724:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55722 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.844748:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55722 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.844756:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55722 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21581824->21585919] (req 21581824->21585919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.844910:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508356352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31769us (32035us total) trans 4294972589 rc 0/0 00000100:00100000:1.0:1789760747.844921:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5301 00000100:00100000:1.0:1789760747.844931:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726851918516 time 726917835340 state 0x2 ref 0 00000100:00100000:3.0:1789760747.849645:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.849745:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508356480 00000100:00100000:0.0:1789760747.849755:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508356480 00000100:00100000:0.0:1789760747.849810:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726851918516 time 726917835340 state 0x0 ref 1 00000100:00100000:0.0:1789760747.849826:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726918095756 time 726917835340 state 0x1 ref 1 00000100:00100000:0.0:1789760747.849853:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726918095756 time 726954564250 state 0x0 ref 1 00000100:00100000:0.0:1789760747.849865:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508356480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.849911:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21585920, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:0.0:1789760747.849922:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55729 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.849939:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760747.849944:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.849950:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55729 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.849964:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55729 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760747.882095:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21585920 -> 21590016 00010000:00010000:0.0:1789760747.882114:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55729 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.882126:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55729 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.882135:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55729 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.882141:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.882147:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55729 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.882156:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55729 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.882180:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55729 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.882188:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55729 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21585920->21590015] (req 21585920->21590015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.882249:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972590 is committed 00000100:00100000:0.0:1789760747.882321:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508356480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32418us (32647us total) trans 4294972590 rc 0/0 00000100:00100000:0.0:1789760747.882332:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5302 00000100:00100000:0.0:1789760747.882344:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726918095756 time 726954564250 state 0x2 ref 0 00000100:00100000:0.0:1789760747.891283:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760747.891495:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508356608 00000100:00100000:2.0:1789760747.891506:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508356608 00000100:00100000:2.0:1789760747.891563:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726918095756 time 726954564250 state 0x0 ref 1 00000100:00100000:2.0:1789760747.891580:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726954824666 time 726954564250 state 0x1 ref 1 00000100:00100000:2.0:1789760747.891610:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726954824666 time 726996320529 state 0x0 ref 1 00000100:00100000:2.0:1789760747.891626:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508356608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760747.891679:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21590016, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:2.0:1789760747.891690:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55730 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.891708:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760747.891713:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760747.891720:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55730 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760747.891736:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55730 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760747.924034:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972591 is committed 00002000:00010000:0.0:1789760747.924181:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21590016 -> 21594112 00010000:00010000:0.0:1789760747.924198:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55730 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.924216:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55730 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.924224:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55730 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.924230:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760747.924235:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55730 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.924245:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55730 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.924269:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55730 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.924278:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55730 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21590016->21594111] (req 21590016->21594111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.924672:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508356608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33041us (33393us total) trans 4294972591 rc 0/0 00000100:00100000:1.0:1789760747.924683:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5303 00000100:00100000:1.0:1789760747.924694:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726954824666 time 726996320529 state 0x2 ref 0 00000100:00100000:2.0:1789760747.931911:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760747.932189:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508356736 00000100:00100000:1.0:1789760747.932202:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508356736 00000100:00100000:1.0:1789760747.933346:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726954824666 time 726996320529 state 0x0 ref 1 00000100:00100000:1.0:1789760747.933366:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726996580945 time 726996320529 state 0x1 ref 1 00000100:00100000:1.0:1789760747.933515:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726996580945 time 727038223602 state 0x0 ref 1 00000100:00100000:1.0:1789760747.933534:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508356736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760747.933589:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21594112, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:1.0:1789760747.933600:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55737 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.933618:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760747.933623:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760747.933631:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55737 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.933647:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55737 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760747.959763:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21594112 -> 21598208 00010000:00010000:1.0:1789760747.959786:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55737 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.959800:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55737 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.959810:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55737 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.959823:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760747.959828:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55737 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.959838:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55737 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.959861:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55737 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760747.959869:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55737 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21594112->21598207] (req 21594112->21598207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760747.959970:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508356736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26437us (28063us total) trans 4294972592 rc 0/0 00000100:00100000:1.0:1789760747.959980:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5304 00000100:00100000:1.0:1789760747.959990:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726996580945 time 727038223602 state 0x2 ref 0 00000001:00080000:0.0:1789760747.960209:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972592 is committed 00000100:00100000:3.0:1789760747.964675:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760747.964844:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508356864 00000100:00100000:0.0:1789760747.964854:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508356864 00000100:00100000:0.0:1789760747.964914:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 726996580945 time 727038223602 state 0x0 ref 1 00000100:00100000:0.0:1789760747.964930:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727038484018 time 727038223602 state 0x1 ref 1 00000100:00100000:0.0:1789760747.965088:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727038484018 time 727069796522 state 0x0 ref 1 00000100:00100000:0.0:1789760747.965102:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508356864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760747.965154:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21598208, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:0.0:1789760747.965164:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5573e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.965181:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760747.965187:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760747.965193:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5573e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760747.965209:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5573e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760748.000519:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21598208 -> 21602304 00010000:00010000:1.0:1789760748.000543:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5573e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.000558:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5573e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.000568:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5573e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.000576:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760748.000581:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5573e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.000597:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5573e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.000621:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5573e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.000630:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5573e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21598208->21602303] (req 21598208->21602303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.000736:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508356864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35630us (36066us total) trans 4294972593 rc 0/0 00000100:00100000:1.0:1789760748.000746:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5305 00000100:00100000:1.0:1789760748.000756:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727038484018 time 727069796522 state 0x2 ref 0 00000001:00080000:3.0:1789760748.000912:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972593 is committed 00000100:00100000:3.0:1789760748.011874:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760748.011959:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508356992 00000100:00100000:1.0:1789760748.011970:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508356992 00000100:00100000:1.0:1789760748.012036:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727038484018 time 727069796522 state 0x0 ref 1 00000100:00100000:1.0:1789760748.012054:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727070056938 time 727069796522 state 0x1 ref 1 00000100:00100000:1.0:1789760748.012092:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727070056938 time 727116801815 state 0x0 ref 1 00000100:00100000:1.0:1789760748.012106:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508356992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760748.012159:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21602304, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:1.0:1789760748.012170:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55745 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.012189:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.012194:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.012201:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55745 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.012218:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55745 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.049365:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972594 is committed 00002000:00010000:1.0:1789760748.049520:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21602304 -> 21606400 00010000:00010000:1.0:1789760748.049537:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55745 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.049551:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55745 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.049561:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55745 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.049568:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760748.049573:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55745 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.049583:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55745 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.049607:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55745 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.049615:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55745 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21602304->21606399] (req 21602304->21606399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.049714:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508356992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37604us (37844us total) trans 4294972594 rc 0/0 00000100:00100000:1.0:1789760748.049725:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5306 00000100:00100000:1.0:1789760748.049736:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727070056938 time 727116801815 state 0x2 ref 0 00000100:00100000:2.0:1789760748.058357:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760748.058500:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508357120 00000100:00100000:1.0:1789760748.058511:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508357120 00000100:00100000:1.0:1789760748.058576:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727070056938 time 727116801815 state 0x0 ref 1 00000100:00100000:1.0:1789760748.058596:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727117062231 time 727116801815 state 0x1 ref 1 00000100:00100000:1.0:1789760748.058630:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727117062231 time 727163340914 state 0x0 ref 1 00000100:00100000:1.0:1789760748.058642:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508357120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760748.058698:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21606400, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:1.0:1789760748.058708:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5574c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.058726:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.058731:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.058737:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5574c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.058757:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5574c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.086174:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972595 is committed 00002000:00010000:2.0:1789760748.086262:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21606400 -> 21610496 00010000:00010000:2.0:1789760748.086278:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5574c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.086308:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5574c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.086318:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5574c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.086330:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760748.086335:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5574c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.086344:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5574c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.086366:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5574c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.086477:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5574c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21606400->21610495] (req 21606400->21610495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.086610:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508357120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27963us (28259us total) trans 4294972595 rc 0/0 00000100:00100000:2.0:1789760748.086621:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5307 00000100:00100000:2.0:1789760748.086631:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727117062231 time 727163340914 state 0x2 ref 0 00000100:00100000:2.0:1789760748.093239:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760748.093529:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508357248 00000100:00100000:1.0:1789760748.093542:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508357248 00000100:00100000:1.0:1789760748.093602:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727117062231 time 727163340914 state 0x0 ref 1 00000100:00100000:1.0:1789760748.093620:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727163601330 time 727163340914 state 0x1 ref 1 00000100:00100000:1.0:1789760748.093693:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727163601330 time 727198401896 state 0x0 ref 1 00000100:00100000:1.0:1789760748.093707:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508357248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760748.093760:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21610496, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:1.0:1789760748.093769:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55753 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.093787:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.093791:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.093798:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55753 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.093814:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55753 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.118170:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972596 is committed 00002000:00010000:0.0:1789760748.118280:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21610496 -> 21614592 00010000:00010000:0.0:1789760748.118313:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55753 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.118326:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55753 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.118336:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55753 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.118349:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760748.118353:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55753 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.118362:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55753 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.118446:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55753 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.118456:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55753 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21610496->21614591] (req 21610496->21614591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.118638:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508357248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24928us (25406us total) trans 4294972596 rc 0/0 00000100:00100000:0.0:1789760748.118650:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5308 00000100:00100000:0.0:1789760748.118662:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727163601330 time 727198401896 state 0x2 ref 0 00000100:00100000:3.0:1789760748.123248:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760748.123548:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508357376 00000100:00100000:1.0:1789760748.123650:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508357376 00000100:00100000:1.0:1789760748.123708:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727163601330 time 727198401896 state 0x0 ref 1 00000100:00100000:1.0:1789760748.123724:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727198662312 time 727198401896 state 0x1 ref 1 00000100:00100000:1.0:1789760748.123782:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727198662312 time 727228493088 state 0x0 ref 1 00000100:00100000:1.0:1789760748.123795:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508357376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760748.123841:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21614592, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:1.0:1789760748.123850:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5575a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.123867:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.123870:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.123877:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5575a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.123891:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5575a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760748.148660:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21614592 -> 21618688 00010000:00010000:2.0:1789760748.148682:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5575a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.148697:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5575a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.148707:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5575a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.148714:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760748.148719:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5575a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.148729:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5575a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.148755:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5575a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.148763:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5575a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21614592->21618687] (req 21614592->21618687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.148854:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508357376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25055us (25610us total) trans 4294972597 rc 0/0 00000100:00100000:2.0:1789760748.148865:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5309 00000100:00100000:2.0:1789760748.148876:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727198662312 time 727228493088 state 0x2 ref 0 00000001:00080000:0.0:1789760748.149111:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972597 is committed 00000100:00100000:2.0:1789760748.157668:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760748.157835:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508357504 00000100:00100000:1.0:1789760748.157846:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508357504 00000100:00100000:1.0:1789760748.157904:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727198662312 time 727228493088 state 0x0 ref 1 00000100:00100000:1.0:1789760748.157920:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727228753504 time 727228493088 state 0x1 ref 1 00000100:00100000:1.0:1789760748.157950:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727228753504 time 727262660751 state 0x0 ref 1 00000100:00100000:1.0:1789760748.157966:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508357504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760748.158018:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21618688, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:1.0:1789760748.158041:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55761 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.158059:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.158063:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.158070:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55761 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.158086:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55761 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.183457:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972598 is committed 00002000:00010000:3.0:1789760748.183766:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21618688 -> 21622784 00010000:00010000:3.0:1789760748.183782:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55761 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.183795:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55761 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.183804:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55761 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.183813:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760748.183819:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55761 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.183828:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55761 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.183851:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55761 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.183860:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55761 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21618688->21622783] (req 21618688->21622783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.183980:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508357504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26010us (26317us total) trans 4294972598 rc 0/0 00000100:00100000:3.0:1789760748.183990:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5310 00000100:00100000:3.0:1789760748.184000:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727228753504 time 727262660751 state 0x2 ref 0 00000100:00100000:2.0:1789760748.188690:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760748.189655:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508357632 00000100:00100000:0.0:1789760748.189668:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508357632 00000100:00100000:0.0:1789760748.189731:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727228753504 time 727262660751 state 0x0 ref 1 00000100:00100000:0.0:1789760748.189749:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727262921167 time 727262660751 state 0x1 ref 1 00000100:00100000:0.0:1789760748.189892:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727262921167 time 727294600357 state 0x0 ref 1 00000100:00100000:0.0:1789760748.189906:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508357632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760748.189969:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21622784, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:0.0:1789760748.189979:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55768 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.189997:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760748.190002:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.190009:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55768 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.190034:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55768 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.218734:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972599 is committed 00002000:00010000:0.0:1789760748.218905:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21622784 -> 21626880 00010000:00010000:0.0:1789760748.218922:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55768 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.218936:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55768 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.218946:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55768 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.218955:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760748.218961:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55768 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.218972:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55768 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.218995:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55768 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.219004:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55768 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21622784->21626879] (req 21622784->21626879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.219101:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508357632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29193us (30416us total) trans 4294972599 rc 0/0 00000100:00100000:0.0:1789760748.219112:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5311 00000100:00100000:0.0:1789760748.219123:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727262921167 time 727294600357 state 0x2 ref 0 00000100:00100000:0.0:1789760748.228051:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760748.228139:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508357760 00000100:00100000:2.0:1789760748.228158:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508357760 00000100:00100000:2.0:1789760748.228228:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727262921167 time 727294600357 state 0x0 ref 1 00000100:00100000:2.0:1789760748.228246:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727294860773 time 727294600357 state 0x1 ref 1 00000100:00100000:2.0:1789760748.228276:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727294860773 time 727332986474 state 0x0 ref 1 00000100:00100000:2.0:1789760748.228325:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508357760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760748.228375:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21626880, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:2.0:1789760748.228391:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5576f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.228408:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.228413:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.228421:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5576f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.228437:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5576f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.269570:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972600 is committed 00002000:00010000:2.0:1789760748.269671:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21626880 -> 21630976 00010000:00010000:2.0:1789760748.269688:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5576f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.269701:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5576f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.269718:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5576f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.269726:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760748.269732:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5576f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.269742:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5576f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.269764:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5576f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.269773:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5576f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21626880->21630975] (req 21626880->21630975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.269872:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508357760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41544us (41825us total) trans 4294972600 rc 0/0 00000100:00100000:2.0:1789760748.269883:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5312 00000100:00100000:2.0:1789760748.269893:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727294860773 time 727332986474 state 0x2 ref 0 00000100:00100000:2.0:1789760748.273273:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760748.273570:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508357888 00000100:00100000:3.0:1789760748.273580:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508357888 00000100:00100000:3.0:1789760748.273639:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727294860773 time 727332986474 state 0x0 ref 1 00000100:00100000:3.0:1789760748.273655:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727333246890 time 727332986474 state 0x1 ref 1 00000100:00100000:3.0:1789760748.273732:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727333246890 time 727378441439 state 0x0 ref 1 00000100:00100000:3.0:1789760748.273748:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508357888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760748.273797:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21630976, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:3.0:1789760748.273807:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55776 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.273824:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760748.273828:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.273835:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55776 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.273850:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55776 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.286524:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789760748.302156:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21630976 -> 21635072 00010000:00010000:3.0:1789760748.302177:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55776 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.302191:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55776 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.302201:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55776 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.302208:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760748.302214:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55776 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.302224:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55776 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.302249:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55776 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.302257:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55776 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21630976->21635071] (req 21630976->21635071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.302412:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508357888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28659us (29144us total) trans 4294972601 rc 0/0 00000100:00100000:3.0:1789760748.302424:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5313 00000100:00100000:3.0:1789760748.302435:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727333246890 time 727378441439 state 0x2 ref 0 00000001:00080000:0.0:1789760748.304716:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972601 is committed 00000100:00100000:0.0:1789760748.307771:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760748.307858:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508358016 00000100:00100000:1.0:1789760748.307869:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508358016 00000100:00100000:1.0:1789760748.307936:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727333246890 time 727378441439 state 0x0 ref 1 00000100:00100000:1.0:1789760748.307957:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727378701855 time 727378441439 state 0x1 ref 1 00000100:00100000:1.0:1789760748.307988:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727378701855 time 727412698506 state 0x0 ref 1 00000100:00100000:1.0:1789760748.308001:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508358016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760748.308062:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21635072, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:1.0:1789760748.308072:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5577d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.308096:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.308101:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.308108:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5577d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.308124:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5577d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760748.339430:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21635072 -> 21639168 00010000:00010000:1.0:1789760748.339451:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5577d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.339465:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5577d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.339474:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5577d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.339481:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760748.339486:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5577d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.339494:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5577d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.339516:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5577d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.339524:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5577d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21635072->21639167] (req 21635072->21639167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.339621:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508358016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31618us (31856us total) trans 4294972602 rc 0/0 00000100:00100000:1.0:1789760748.339632:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5314 00000100:00100000:1.0:1789760748.339643:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727378701855 time 727412698506 state 0x2 ref 0 00000001:00080000:0.0:1789760748.340138:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972602 is committed 00000100:00100000:2.0:1789760748.344461:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760748.345462:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508358144 00000100:00100000:1.0:1789760748.345475:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508358144 00000100:00100000:1.0:1789760748.345539:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727378701855 time 727412698506 state 0x0 ref 1 00000100:00100000:1.0:1789760748.345557:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727412958922 time 727412698506 state 0x1 ref 1 00000100:00100000:1.0:1789760748.345591:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727412958922 time 727450299991 state 0x0 ref 1 00000100:00100000:1.0:1789760748.345605:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508358144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760748.345658:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21639168, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:1.0:1789760748.345674:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55784 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.345691:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.346052:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.346063:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55784 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.346086:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55784 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.382855:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972603 is committed 00002000:00010000:0.0:1789760748.385543:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21639168 -> 21643264 00010000:00010000:0.0:1789760748.385564:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55784 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.385577:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55784 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.385586:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55784 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.385594:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760748.385599:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55784 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.385607:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55784 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.385693:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55784 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.385701:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe55784 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21639168->21643263] (req 21639168->21643263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.386014:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508358144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40406us (41557us total) trans 4294972603 rc 0/0 00000100:00100000:1.0:1789760748.386035:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5315 00000100:00100000:1.0:1789760748.386046:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727412958922 time 727450299991 state 0x2 ref 0 00000100:00100000:3.0:1789760748.390089:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760748.390164:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508358272 00000100:00100000:2.0:1789760748.390175:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508358272 00000100:00100000:2.0:1789760748.390234:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727412958922 time 727450299991 state 0x0 ref 1 00000100:00100000:2.0:1789760748.390250:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727450560407 time 727450299991 state 0x1 ref 1 00000100:00100000:2.0:1789760748.390280:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727450560407 time 727494990267 state 0x0 ref 1 00000100:00100000:2.0:1789760748.390333:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508358272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760748.390384:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21643264, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:2.0:1789760748.390395:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5578b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.390412:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.390417:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.390424:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5578b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.390439:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5578b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.444700:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972604 is committed 00002000:00010000:0.0:1789760748.444804:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21643264 -> 21647360 00010000:00010000:0.0:1789760748.444819:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5578b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.444831:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5578b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.444840:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5578b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.444851:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760748.444857:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5578b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.444865:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5578b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.444888:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5578b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.444896:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5578b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21643264->21647359] (req 21643264->21647359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.444998:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508358272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54663us (54915us total) trans 4294972604 rc 0/0 00000100:00100000:0.0:1789760748.445008:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5316 00000100:00100000:0.0:1789760748.445018:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727450560407 time 727494990267 state 0x2 ref 0 00000100:00100000:0.0:1789760748.449228:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760748.449321:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508358400 00000100:00100000:1.0:1789760748.449332:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508358400 00000100:00100000:1.0:1789760748.449401:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727450560407 time 727494990267 state 0x0 ref 1 00000100:00100000:1.0:1789760748.449426:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727495250683 time 727494990267 state 0x1 ref 1 00000100:00100000:1.0:1789760748.449452:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727495250683 time 727554162187 state 0x0 ref 1 00000100:00100000:1.0:1789760748.449464:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508358400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760748.449514:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21647360, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:1.0:1789760748.449523:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55792 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.449540:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.449545:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.449552:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55792 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.449566:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55792 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.483633:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972605 is committed 00002000:00010000:1.0:1789760748.483871:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21647360 -> 21651456 00010000:00010000:1.0:1789760748.483888:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55792 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.483902:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55792 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.483910:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55792 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.483917:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760748.483922:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55792 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.483931:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55792 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.484034:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55792 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.484042:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55792 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21647360->21651455] (req 21647360->21651455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760748.484131:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508358400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34663us (34911us total) trans 4294972605 rc 0/0 00000100:00100000:1.0:1789760748.484142:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5317 00000100:00100000:1.0:1789760748.484153:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727495250683 time 727554162187 state 0x2 ref 0 00000100:00100000:2.0:1789760748.492415:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760748.492748:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508358528 00000100:00100000:1.0:1789760748.492757:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508358528 00000100:00100000:1.0:1789760748.492813:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727495250683 time 727554162187 state 0x0 ref 1 00000100:00100000:1.0:1789760748.492827:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727554422603 time 727554162187 state 0x1 ref 1 00000100:00100000:1.0:1789760748.492854:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727554422603 time 727597565329 state 0x0 ref 1 00000100:00100000:1.0:1789760748.492866:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508358528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760748.492912:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21651456, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0xa807 00010000:00010000:1.0:1789760748.492921:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55799 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.492949:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.492953:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.492961:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55799 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.492975:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55799 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.520947:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972606 is committed 00002000:00010000:2.0:1789760748.521260:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21651456 -> 21655552 00002000:00010000:2.0:1789760748.521265:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760748 -> 1789760749 00002000:00010000:2.0:1789760748.521267:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760748 -> 1789760749 00010000:00010000:2.0:1789760748.521282:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55799 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.521308:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55799 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.521318:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55799 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.521327:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760748.521332:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55799 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.521343:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55799 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.521365:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55799 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.521385:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55799 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21651456->21655551] (req 21651456->21655551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.521484:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508358528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28613us (29073us total) trans 4294972606 rc 0/0 00000100:00100000:2.0:1789760748.521495:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5318 00000100:00100000:2.0:1789760748.521505:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727554422603 time 727597565329 state 0x2 ref 0 00000100:00100000:3.0:1789760748.525172:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760748.525743:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508358656 00000100:00100000:0.0:1789760748.525754:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508358656 00000100:00100000:0.0:1789760748.525814:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727554422603 time 727597565329 state 0x0 ref 1 00000100:00100000:0.0:1789760748.525832:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727597825745 time 727597565329 state 0x1 ref 1 00000100:00100000:0.0:1789760748.525867:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727597825745 time 727630577422 state 0x0 ref 1 00000100:00100000:0.0:1789760748.525881:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508358656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760748.525931:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21655552, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:0.0:1789760748.525952:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe557a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.525969:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760748.525974:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.525981:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe557a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.525997:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe557a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.552770:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972607 is committed 00002000:00010000:2.0:1789760748.552871:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21655552 -> 21659648 00010000:00010000:2.0:1789760748.552887:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe557a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.552906:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe557a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.552915:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe557a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.552923:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760748.552928:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe557a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.552938:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe557a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.552960:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe557a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.552967:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe557a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21655552->21659647] (req 21655552->21659647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.553074:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508358656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27189us (27908us total) trans 4294972607 rc 0/0 00000100:00100000:2.0:1789760748.553085:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5319 00000100:00100000:2.0:1789760748.553095:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727597825745 time 727630577422 state 0x2 ref 0 00000100:00100000:0.0:1789760748.556616:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760748.556706:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508358784 00000100:00100000:1.0:1789760748.556717:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508358784 00000100:00100000:1.0:1789760748.556773:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727597825745 time 727630577422 state 0x0 ref 1 00000100:00100000:1.0:1789760748.556790:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727630837838 time 727630577422 state 0x1 ref 1 00000100:00100000:1.0:1789760748.556821:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727630837838 time 727661531529 state 0x0 ref 1 00000100:00100000:1.0:1789760748.556835:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508358784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760748.556887:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21659648, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:1.0:1789760748.556897:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe557a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.556914:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760748.556918:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760748.556926:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe557a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760748.556941:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe557a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.600815:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972608 is committed 00002000:00010000:2.0:1789760748.601175:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21659648 -> 21663744 00010000:00010000:2.0:1789760748.601193:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe557a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.601206:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe557a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.601217:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe557a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.601225:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760748.601231:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe557a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.601241:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe557a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.601264:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe557a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.601272:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe557a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21659648->21663743] (req 21659648->21663743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.601403:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508358784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44565us (44796us total) trans 4294972608 rc 0/0 00000100:00100000:2.0:1789760748.601415:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5320 00000100:00100000:2.0:1789760748.601427:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727630837838 time 727661531529 state 0x2 ref 0 00000100:00100000:3.0:1789760748.604216:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760748.604344:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508358912 00000100:00100000:2.0:1789760748.604355:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508358912 00000100:00100000:2.0:1789760748.604414:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727630837838 time 727661531529 state 0x0 ref 1 00000100:00100000:2.0:1789760748.604431:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727661791945 time 727661531529 state 0x1 ref 1 00000100:00100000:2.0:1789760748.604524:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727661791945 time 727709233208 state 0x0 ref 1 00000100:00100000:2.0:1789760748.604539:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508358912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760748.604596:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21663744, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760748.604606:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.604624:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.604629:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.604637:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.604652:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760748.683125:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21663744 -> 21667840 00010000:00010000:2.0:1789760748.683147:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.683160:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.683169:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.683176:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760748.683181:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.683190:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.683214:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.683222:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21663744->21667839] (req 21663744->21667839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.683369:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508358912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 78826us (79160us total) trans 4294972609 rc 0/0 00000100:00100000:2.0:1789760748.683380:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5321 00000100:00100000:2.0:1789760748.683390:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727661791945 time 727709233208 state 0x2 ref 0 00000001:00080000:1.0:1789760748.683473:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972609 is committed 00000100:00100000:3.0:1789760748.691497:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760748.692054:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508359040 00000100:00100000:2.0:1789760748.692066:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508359040 00000100:00100000:2.0:1789760748.692125:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727661791945 time 727709233208 state 0x0 ref 1 00000100:00100000:2.0:1789760748.692141:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727709493624 time 727709233208 state 0x1 ref 1 00000100:00100000:2.0:1789760748.692178:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727709493624 time 727796888475 state 0x0 ref 1 00000100:00100000:2.0:1789760748.692190:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508359040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760748.692243:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21667840, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760748.692253:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.692269:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.692274:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.692281:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.692336:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.750555:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972610 is committed 00002000:00010000:0.0:1789760748.750600:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21667840 -> 21671936 00010000:00010000:0.0:1789760748.750616:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.750629:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.750636:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.750640:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760748.750644:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.750714:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.750739:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.750744:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe557b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21667840->21671935] (req 21667840->21671935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760748.750832:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508359040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58638us (59338us total) trans 4294972610 rc 0/0 00000100:00100000:0.0:1789760748.750842:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5322 00000100:00100000:0.0:1789760748.750853:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727709493624 time 727796888475 state 0x2 ref 0 00000100:00100000:0.0:1789760748.760248:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760748.760396:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508359168 00000100:00100000:2.0:1789760748.760407:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508359168 00000100:00100000:2.0:1789760748.760466:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727709493624 time 727796888475 state 0x0 ref 1 00000100:00100000:2.0:1789760748.760485:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727797148891 time 727796888475 state 0x1 ref 1 00000100:00100000:2.0:1789760748.760513:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727797148891 time 727865223552 state 0x0 ref 1 00000100:00100000:2.0:1789760748.760527:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508359168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760748.760579:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21671936, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760748.760589:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe557bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.760604:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.760609:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.760616:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe557bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.760630:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe557bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760748.791871:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972611 is committed 00002000:00010000:3.0:1789760748.791955:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21671936 -> 21676032 00010000:00010000:3.0:1789760748.791971:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe557bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.791984:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe557bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.791994:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe557bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.792001:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760748.792134:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe557bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.792147:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe557bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.792172:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe557bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.792181:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe557bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21671936->21676031] (req 21671936->21676031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.792353:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508359168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31823us (32110us total) trans 4294972611 rc 0/0 00000100:00100000:3.0:1789760748.792364:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5323 00000100:00100000:3.0:1789760748.792384:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727797148891 time 727865223552 state 0x2 ref 0 00000100:00100000:3.0:1789760748.799518:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760748.799639:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508359296 00000100:00100000:2.0:1789760748.799650:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508359296 00000100:00100000:2.0:1789760748.799707:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727797148891 time 727865223552 state 0x0 ref 1 00000100:00100000:2.0:1789760748.799724:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727865483968 time 727865223552 state 0x1 ref 1 00000100:00100000:2.0:1789760748.799791:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727865483968 time 727904500433 state 0x0 ref 1 00000100:00100000:2.0:1789760748.799806:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508359296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760748.799862:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21676032, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760748.799873:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe557c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.799892:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760748.799896:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760748.799904:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe557c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760748.799921:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe557c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760748.831958:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972612 is committed 00002000:00010000:3.0:1789760748.832083:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21676032 -> 21680128 00010000:00010000:3.0:1789760748.832100:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe557c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.832120:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe557c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.832128:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe557c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.832135:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760748.832140:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe557c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.832149:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe557c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.832173:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe557c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.832182:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe557c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21676032->21680127] (req 21676032->21680127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.832279:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508359296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32471us (32765us total) trans 4294972612 rc 0/0 00000100:00100000:3.0:1789760748.832325:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5324 00000100:00100000:3.0:1789760748.832336:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727865483968 time 727904500433 state 0x2 ref 0 00000100:00100000:0.0:1789760748.836801:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760748.837193:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508359424 00000100:00100000:2.0:1789760748.837205:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508359424 00000100:00100000:2.0:1789760748.837263:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727865483968 time 727904500433 state 0x0 ref 1 00000100:00100000:2.0:1789760748.837282:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727904760849 time 727904500433 state 0x1 ref 1 00000100:00100000:3.0:1789760748.838334:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727904760849 time 727943039083 state 0x0 ref 1 00000100:00100000:3.0:1789760748.838358:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508359424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760748.838416:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21680128, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:3.0:1789760748.838426:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.838446:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760748.838451:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.838459:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.838475:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.870394:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21680128 -> 21684224 00010000:00010000:3.0:1789760748.870413:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870426:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870436:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870444:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760748.870456:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870467:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870489:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.870496:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21680128->21684223] (req 21680128->21684223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.870578:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508359424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32225us (33782us total) trans 4294972613 rc 0/0 00000100:00100000:3.0:1789760748.870588:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5325 00000100:00100000:3.0:1789760748.870597:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727904760849 time 727943039083 state 0x2 ref 0 00000001:00080000:3.0:1789760748.870633:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972613 is committed 00000100:00100000:1.0:1789760748.876511:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760748.876712:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508359552 00000100:00100000:3.0:1789760748.876724:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508359552 00000100:00100000:3.0:1789760748.876784:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727904760849 time 727943039083 state 0x0 ref 1 00000100:00100000:3.0:1789760748.876802:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727943299499 time 727943039083 state 0x1 ref 1 00000100:00100000:3.0:1789760748.876835:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727943299499 time 727981544623 state 0x0 ref 1 00000100:00100000:3.0:1789760748.876847:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508359552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760748.876897:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21684224, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:3.0:1789760748.876909:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.876928:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760748.876994:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.877003:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.877052:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.922625:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972614 is committed 00002000:00010000:3.0:1789760748.922700:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21684224 -> 21688320 00010000:00010000:3.0:1789760748.922717:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.922731:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.922741:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.922749:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760748.922754:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.922764:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.922791:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.922800:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe557d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21684224->21688319] (req 21684224->21688319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.922919:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508359552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46068us (46414us total) trans 4294972614 rc 0/0 00000100:00100000:3.0:1789760748.922929:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5326 00000100:00100000:3.0:1789760748.922939:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727943299499 time 727981544623 state 0x2 ref 0 00000100:00100000:3.0:1789760748.933266:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760748.933337:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508359680 00000100:00100000:2.0:1789760748.933350:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508359680 00000100:00100000:2.0:1789760748.933414:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727943299499 time 727981544623 state 0x0 ref 1 00000100:00100000:2.0:1789760748.933431:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727981805039 time 727981544623 state 0x1 ref 1 00000100:00100000:0.0:1789760748.933605:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727981805039 time 728038313144 state 0x0 ref 1 00000100:00100000:0.0:1789760748.933619:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508359680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760748.933670:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21688320, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:0.0:1789760748.933680:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe557d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.933698:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760748.933703:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760748.933710:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe557d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760748.933726:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe557d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760748.988338:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972615 is committed 00002000:00010000:3.0:1789760748.989231:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21688320 -> 21692416 00010000:00010000:3.0:1789760748.989250:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe557d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.989264:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe557d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.989273:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe557d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.989281:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760748.989316:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe557d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.989330:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe557d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.989356:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe557d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.989365:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe557d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21688320->21692415] (req 21688320->21692415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760748.993183:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508359680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59552us (59912us total) trans 4294972615 rc 0/0 00000100:00100000:2.0:1789760748.993200:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5327 00000100:00100000:2.0:1789760748.993213:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727981805039 time 728038313144 state 0x2 ref 0 00000100:00100000:0.0:1789760749.003769:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760749.004184:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508359808 00000100:00100000:3.0:1789760749.004195:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508359808 00000100:00080000:0.0:1789760749.004530:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760749.004534:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 727981805039 time 728038313144 state 0x0 ref 1 00000100:00100000:2.0:1789760749.004553:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728038573560 time 728038313144 state 0x1 ref 1 00000100:00100000:2.0:1789760749.004792:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728038573560 time 728109500633 state 0x0 ref 1 00000100:00100000:2.0:1789760749.004814:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508359808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.004871:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21692416, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760749.004881:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.004899:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.004904:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.004912:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.004928:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760749.050168:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21692416 -> 21696512 00010000:00010000:2.0:1789760749.050188:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.050202:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.050211:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.050218:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.050224:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.050234:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.050256:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.050263:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21692416->21696511] (req 21692416->21696511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.050445:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508359808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45632us (46794us total) trans 4294972616 rc 0/0 00000100:00100000:2.0:1789760749.050455:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5328 00000100:00100000:2.0:1789760749.050466:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728038573560 time 728109500633 state 0x2 ref 0 00000001:00080000:1.0:1789760749.050530:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972616 is committed 00000100:00100000:0.0:1789760749.053874:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760749.054035:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508359936 00000100:00100000:3.0:1789760749.054046:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508359936 00000100:00100000:3.0:1789760749.054106:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728038573560 time 728109500633 state 0x0 ref 1 00000100:00100000:3.0:1789760749.054123:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728109761049 time 728109500633 state 0x1 ref 1 00000100:00100000:3.0:1789760749.054188:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728109761049 time 728158898579 state 0x0 ref 1 00000100:00100000:3.0:1789760749.054203:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d43050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508359936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760749.054254:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21696512, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:3.0:1789760749.054264:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe557e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.054361:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760749.054488:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.054496:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe557e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.054512:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe557e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.082829:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972617 is committed 00002000:00010000:2.0:1789760749.083510:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21696512 -> 21700608 00010000:00010000:2.0:1789760749.083528:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe557e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.083541:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe557e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.083550:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe557e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.083556:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.083562:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe557e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.083570:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe557e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.083594:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe557e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.083602:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe557e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21696512->21700607] (req 21696512->21700607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.083761:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d43050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508359936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29556us (29890us total) trans 4294972617 rc 0/0 00000100:00100000:2.0:1789760749.083772:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5329 00000100:00100000:2.0:1789760749.083782:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728109761049 time 728158898579 state 0x2 ref 0 00000100:00100000:3.0:1789760749.087843:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.087999:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508360064 00000100:00100000:2.0:1789760749.088010:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508360064 00000100:00100000:2.0:1789760749.088081:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728109761049 time 728158898579 state 0x0 ref 1 00000100:00100000:2.0:1789760749.088097:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728159158995 time 728158898579 state 0x1 ref 1 00000100:00100000:2.0:1789760749.088174:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728159158995 time 728192882867 state 0x0 ref 1 00000100:00100000:2.0:1789760749.088188:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508360064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.088240:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21700608, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760749.088251:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.088268:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.088274:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.088281:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.088318:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.120245:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972618 is committed 00002000:00010000:2.0:1789760749.120360:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21700608 -> 21704704 00010000:00010000:2.0:1789760749.120388:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.120402:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.120412:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.120419:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.120424:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.120435:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.120465:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.120473:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21700608->21704703] (req 21700608->21704703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.120573:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508360064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32382us (32735us total) trans 4294972618 rc 0/0 00000100:00100000:2.0:1789760749.120584:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5330 00000100:00100000:2.0:1789760749.120594:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728159158995 time 728192882867 state 0x2 ref 0 00000100:00100000:0.0:1789760749.124391:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.124455:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508360192 00000100:00100000:2.0:1789760749.124466:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508360192 00000100:00100000:2.0:1789760749.124525:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728159158995 time 728192882867 state 0x0 ref 1 00000100:00100000:2.0:1789760749.124543:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728193143283 time 728192882867 state 0x1 ref 1 00000100:00100000:2.0:1789760749.124575:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728193143283 time 728229285635 state 0x0 ref 1 00000100:00100000:2.0:1789760749.124588:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508360192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.124711:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21704704, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760749.124721:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe557f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.124739:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.124744:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.124752:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe557f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.124767:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe557f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.159277:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972619 is committed 00002000:00010000:2.0:1789760749.159613:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21704704 -> 21708800 00010000:00010000:2.0:1789760749.159629:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe557f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.159643:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe557f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.159653:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe557f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.159660:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.159665:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe557f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.159675:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe557f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.159699:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe557f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.159707:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe557f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21704704->21708799] (req 21704704->21708799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.159842:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508360192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35250us (35455us total) trans 4294972619 rc 0/0 00000100:00100000:2.0:1789760749.159852:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5331 00000100:00100000:2.0:1789760749.159863:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728193143283 time 728229285635 state 0x2 ref 0 00000100:00100000:3.0:1789760749.163499:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.163610:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508360320 00000100:00100000:2.0:1789760749.163620:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508360320 00000100:00100000:2.0:1789760749.163676:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728193143283 time 728229285635 state 0x0 ref 1 00000100:00100000:2.0:1789760749.163693:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728229546051 time 728229285635 state 0x1 ref 1 00000100:00100000:2.0:1789760749.163715:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728229546051 time 728268425209 state 0x0 ref 1 00000100:00100000:2.0:1789760749.163727:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508360320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.163777:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21708800, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760749.163786:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.163804:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.163808:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.163815:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.163829:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.195925:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972620 is committed 00002000:00010000:2.0:1789760749.196112:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21708800 -> 21712896 00010000:00010000:2.0:1789760749.196130:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.196144:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.196153:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.196161:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.196166:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.196175:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.196200:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.196208:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe557fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21708800->21712895] (req 21708800->21712895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.196340:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508360320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32610us (32844us total) trans 4294972620 rc 0/0 00000100:00100000:2.0:1789760749.196355:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5332 00000100:00100000:2.0:1789760749.196376:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728229546051 time 728268425209 state 0x2 ref 0 00000100:00100000:3.0:1789760749.200355:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.200459:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508360448 00000100:00100000:2.0:1789760749.200469:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508360448 00000100:00100000:2.0:1789760749.200528:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728229546051 time 728268425209 state 0x0 ref 1 00000100:00100000:2.0:1789760749.200544:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728268685625 time 728268425209 state 0x1 ref 1 00000100:00100000:2.0:1789760749.200564:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728268685625 time 728305275404 state 0x0 ref 1 00000100:00100000:2.0:1789760749.200576:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508360448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.200628:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21712896, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760749.200638:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55802 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.200655:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.200659:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.200666:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55802 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.200681:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55802 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.240674:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972621 is committed 00002000:00010000:2.0:1789760749.242864:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21712896 -> 21716992 00010000:00010000:2.0:1789760749.242884:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55802 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.242897:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55802 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.242904:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55802 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.242910:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.242915:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55802 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.242924:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55802 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.242947:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55802 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.242954:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55802 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21712896->21716991] (req 21712896->21716991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.243188:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508360448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42606us (42836us total) trans 4294972621 rc 0/0 00000100:00100000:2.0:1789760749.243200:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5333 00000100:00100000:2.0:1789760749.243210:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728268685625 time 728305275404 state 0x2 ref 0 00000100:00100000:0.0:1789760749.253794:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.253892:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508360576 00000100:00100000:2.0:1789760749.253902:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508360576 00000100:00100000:2.0:1789760749.254041:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728268685625 time 728305275404 state 0x0 ref 1 00000100:00100000:2.0:1789760749.254059:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728305535820 time 728305275404 state 0x1 ref 1 00000100:00100000:2.0:1789760749.254124:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728305535820 time 728358833049 state 0x0 ref 1 00000100:00100000:2.0:1789760749.254138:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508360576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.254190:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21716992, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760749.254200:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55809 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.254216:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.254221:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.254228:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55809 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.254244:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55809 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760749.290108:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21716992 -> 21721088 00010000:00010000:2.0:1789760749.290142:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55809 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.290156:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55809 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.290172:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55809 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.290179:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.290184:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55809 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.290194:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55809 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.290220:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55809 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.290229:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55809 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21716992->21721087] (req 21716992->21721087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.290411:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508360576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36270us (36621us total) trans 4294972622 rc 0/0 00000100:00100000:2.0:1789760749.290423:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5334 00000100:00100000:2.0:1789760749.290433:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728305535820 time 728358833049 state 0x2 ref 0 00000001:00080000:1.0:1789760749.290675:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972622 is committed 00000100:00100000:3.0:1789760749.303883:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.304010:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508360704 00000100:00100000:2.0:1789760749.304031:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508360704 00000100:00100000:2.0:1789760749.304103:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728305535820 time 728358833049 state 0x0 ref 1 00000100:00100000:2.0:1789760749.304122:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728359093465 time 728358833049 state 0x1 ref 1 00000100:00100000:2.0:1789760749.304157:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728359093465 time 728408867341 state 0x0 ref 1 00000100:00100000:2.0:1789760749.304171:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508360704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.304230:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21721088, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760749.304240:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55810 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.304257:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.304262:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.304270:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55810 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.304321:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55810 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760749.310637:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760749.330082:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972623 is committed 00002000:00010000:2.0:1789760749.330155:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21721088 -> 21725184 00010000:00010000:2.0:1789760749.330171:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55810 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.330184:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55810 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.330194:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55810 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.330201:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.330206:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55810 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.330216:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55810 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.330237:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55810 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.330243:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55810 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21721088->21725183] (req 21721088->21725183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.330534:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508360704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26361us (26655us total) trans 4294972623 rc 0/0 00000100:00100000:2.0:1789760749.330545:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5335 00000100:00100000:2.0:1789760749.330556:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728359093465 time 728408867341 state 0x2 ref 0 00000100:00100000:0.0:1789760749.335030:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.335124:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508360832 00000100:00100000:2.0:1789760749.335134:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508360832 00000100:00100000:2.0:1789760749.335194:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728359093465 time 728408867341 state 0x0 ref 1 00000100:00100000:2.0:1789760749.335213:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728409127757 time 728408867341 state 0x1 ref 1 00000100:00100000:2.0:1789760749.335311:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728409127757 time 728439993296 state 0x0 ref 1 00000100:00100000:2.0:1789760749.335327:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508360832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.335390:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21725184, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760749.335401:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55817 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.335418:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.335423:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.335430:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55817 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.335446:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55817 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.380708:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21725184 -> 21729280 00010000:00010000:3.0:1789760749.380728:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55817 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.380740:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55817 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.380748:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55817 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.380755:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760749.380760:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55817 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.380774:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55817 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.380796:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55817 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.380803:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55817 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21725184->21729279] (req 21725184->21729279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.380903:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508360832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45574us (45887us total) trans 4294972624 rc 0/0 00000100:00100000:3.0:1789760749.380914:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5336 00000100:00100000:3.0:1789760749.380925:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728409127757 time 728439993296 state 0x2 ref 0 00000001:00080000:1.0:1789760749.384069:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972624 is committed 00000100:00100000:3.0:1789760749.384838:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.384922:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508360960 00000100:00100000:2.0:1789760749.384941:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508360960 00000100:00100000:2.0:1789760749.384999:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728409127757 time 728439993296 state 0x0 ref 1 00000100:00100000:2.0:1789760749.385028:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728440253712 time 728439993296 state 0x1 ref 1 00000100:00100000:2.0:1789760749.385060:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728440253712 time 728489770614 state 0x0 ref 1 00000100:00100000:2.0:1789760749.385074:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508360960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.385125:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21729280, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760749.385135:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5581e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.385153:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.385158:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.385165:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5581e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.385182:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5581e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760749.426888:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21729280 -> 21733376 00010000:00010000:2.0:1789760749.426910:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5581e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.426924:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5581e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.426944:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5581e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.426952:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.426956:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5581e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.426966:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5581e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.426990:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5581e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.426998:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5581e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21729280->21733375] (req 21729280->21733375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.427108:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508360960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42031us (42271us total) trans 4294972625 rc 0/0 00000100:00100000:2.0:1789760749.427120:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5337 00000100:00100000:2.0:1789760749.427131:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728440253712 time 728489770614 state 0x2 ref 0 00000001:00080000:1.0:1789760749.427270:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972625 is committed 00000100:00100000:2.0:1789760749.430726:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760749.430801:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508361088 00000100:00100000:3.0:1789760749.430812:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508361088 00000100:00100000:3.0:1789760749.430869:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728440253712 time 728489770614 state 0x0 ref 1 00000100:00100000:3.0:1789760749.430884:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728490031030 time 728489770614 state 0x1 ref 1 00000100:00100000:3.0:1789760749.430913:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728490031030 time 728535624224 state 0x0 ref 1 00000100:00100000:3.0:1789760749.430932:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508361088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760749.430978:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21733376, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:3.0:1789760749.430987:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55825 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.431004:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760749.431009:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.431024:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55825 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.431039:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55825 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.439670:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048840c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534073088:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760749.439742:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760749.439901:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534073088 00000100:00100000:1.0:1789760749.439910:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534073088 00000100:00100000:1.0:1789760749.440053:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:1.0:1789760749.440064:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534073088:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760749.440106:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760749.440322:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534073088:12345-0@lo:41:osp-pre-1-0.0 Request processed in 259us (579us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760749.440330:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00100000:3.0:1789760749.440564:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048840c0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534073088:0@lo:41:osp-pre-1-0.0 00000001:00080000:3.0:1789760749.459115:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972626 is committed 00002000:00010000:2.0:1789760749.459223:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21733376 -> 21737472 00010000:00010000:2.0:1789760749.459240:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55825 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.459251:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55825 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.459260:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55825 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.459267:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.459271:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55825 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.459280:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55825 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.459334:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55825 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.459347:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55825 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21733376->21737471] (req 21733376->21737471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.459445:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508361088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28515us (28723us total) trans 4294972626 rc 0/0 00000100:00100000:2.0:1789760749.459456:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5338 00000100:00100000:2.0:1789760749.459467:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728490031030 time 728535624224 state 0x2 ref 0 00000100:00100000:3.0:1789760749.464276:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.464418:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508361216 00000100:00100000:2.0:1789760749.464429:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508361216 00000100:00100000:2.0:1789760749.464485:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728490031030 time 728535624224 state 0x0 ref 1 00000100:00100000:2.0:1789760749.464501:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728535884640 time 728535624224 state 0x1 ref 1 00000100:00100000:2.0:1789760749.464530:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728535884640 time 728569240665 state 0x0 ref 1 00000100:00100000:2.0:1789760749.464542:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508361216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.464592:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21737472, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760749.464602:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5582c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.464620:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.464625:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.464633:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5582c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.464654:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5582c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.512807:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972627 is committed 00002000:00010000:3.0:1789760749.513150:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21737472 -> 21741568 00010000:00010000:3.0:1789760749.513166:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5582c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.513177:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5582c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.513185:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5582c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.513192:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760749.513197:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5582c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.513211:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5582c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.513233:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5582c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.513240:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5582c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21737472->21741567] (req 21737472->21741567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.513403:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508361216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48856us (49132us total) trans 4294972627 rc 0/0 00000100:00100000:3.0:1789760749.513414:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5339 00000100:00100000:3.0:1789760749.513425:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728535884640 time 728569240665 state 0x2 ref 0 00000100:00100000:3.0:1789760749.520434:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.520696:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508361344 00000100:00100000:2.0:1789760749.520707:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508361344 00000100:00100000:2.0:1789760749.520765:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728535884640 time 728569240665 state 0x0 ref 1 00000100:00100000:2.0:1789760749.520787:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728569501081 time 728569240665 state 0x1 ref 1 00000100:00100000:2.0:1789760749.520823:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728569501081 time 728625533923 state 0x0 ref 1 00000100:00100000:2.0:1789760749.520834:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508361344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.520892:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21741568, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0xa807 00010000:00010000:2.0:1789760749.520901:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55833 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.520917:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.520921:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.520927:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55833 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.520941:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55833 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.570107:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972628 is committed 00002000:00010000:3.0:1789760749.570396:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21741568 -> 21745664 00002000:00010000:3.0:1789760749.570401:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760749 -> 1789760750 00002000:00010000:3.0:1789760749.570404:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760749 -> 1789760750 00010000:00010000:3.0:1789760749.570419:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55833 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.570434:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55833 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.570444:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55833 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.570452:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760749.570458:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55833 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.570468:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55833 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.570491:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55833 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.570499:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55833 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21741568->21745663] (req 21741568->21745663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.570602:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508361344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49763us (50172us total) trans 4294972628 rc 0/0 00000100:00100000:3.0:1789760749.570612:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5340 00000100:00100000:3.0:1789760749.570623:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728569501081 time 728625533923 state 0x2 ref 0 00000100:00100000:2.0:1789760749.576123:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760749.576332:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508361472 00000100:00100000:3.0:1789760749.576344:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508361472 00000100:00100000:2.0:1789760749.576463:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728569501081 time 728625533923 state 0x0 ref 1 00000100:00100000:2.0:1789760749.576482:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728625794339 time 728625533923 state 0x1 ref 1 00000100:00100000:2.0:1789760749.576512:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728625794339 time 728681222691 state 0x0 ref 1 00000100:00100000:2.0:1789760749.576526:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508361472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.576578:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21745664, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:2.0:1789760749.576588:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5583a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.576604:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.576608:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.576615:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5583a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.576630:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5583a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760749.646505:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21745664 -> 21749760 00010000:00010000:2.0:1789760749.646528:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5583a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.646543:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5583a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.646620:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5583a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.646628:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.646633:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5583a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.646903:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972629 is committed 00010000:00010000:2.0:1789760749.649057:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5583a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.649088:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5583a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.649107:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5583a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21745664->21749759] (req 21745664->21749759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.653382:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508361472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 76844us (77254us total) trans 4294972629 rc 0/0 00000100:00100000:2.0:1789760749.653397:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5341 00000100:00100000:2.0:1789760749.653408:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728625794339 time 728681222691 state 0x2 ref 0 00000100:00100000:0.0:1789760749.661933:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760749.662032:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508361600 00000100:00100000:3.0:1789760749.662043:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508361600 00000100:00100000:3.0:1789760749.662263:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728625794339 time 728681222691 state 0x0 ref 1 00000100:00100000:3.0:1789760749.662282:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728681483107 time 728681222691 state 0x1 ref 1 00000100:00100000:3.0:1789760749.662359:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728681483107 time 728767069417 state 0x0 ref 1 00000100:00100000:3.0:1789760749.662374:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508361600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760749.662426:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21749760, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:3.0:1789760749.662436:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55841 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.662453:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760749.662458:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.662466:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55841 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.662482:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55841 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.713961:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972630 is committed 00002000:00010000:3.0:1789760749.714150:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21749760 -> 21753856 00010000:00010000:3.0:1789760749.714167:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55841 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.714179:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55841 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.714188:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55841 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.714195:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760749.714201:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55841 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.714210:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55841 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.714233:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55841 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.714241:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55841 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21749760->21753855] (req 21749760->21753855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.714426:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508361600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52049us (52497us total) trans 4294972630 rc 0/0 00000100:00100000:3.0:1789760749.714437:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5342 00000100:00100000:3.0:1789760749.714448:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728681483107 time 728767069417 state 0x2 ref 0 00000100:00100000:3.0:1789760749.718048:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.718151:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508361728 00000100:00100000:2.0:1789760749.718162:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508361728 00000100:00100000:2.0:1789760749.718220:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728681483107 time 728767069417 state 0x0 ref 1 00000100:00100000:2.0:1789760749.718238:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728767329833 time 728767069417 state 0x1 ref 1 00000100:00100000:2.0:1789760749.718269:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728767329833 time 728822979828 state 0x0 ref 1 00000100:00100000:2.0:1789760749.718323:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508361728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.718374:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21753856, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:2.0:1789760749.718383:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55848 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.718400:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.718405:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.718411:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55848 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.718426:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55848 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760749.757581:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21753856 -> 21757952 00010000:00010000:2.0:1789760749.757605:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55848 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.757619:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55848 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.757630:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55848 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.757638:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.757643:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55848 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.757655:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55848 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.757679:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55848 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.757687:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55848 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21753856->21757951] (req 21753856->21757951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.757790:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508361728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39502us (39748us total) trans 4294972631 rc 0/0 00000100:00100000:2.0:1789760749.757801:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5343 00000100:00100000:2.0:1789760749.757812:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728767329833 time 728822979828 state 0x2 ref 0 00000001:00080000:3.0:1789760749.757895:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972631 is committed 00000100:00100000:2.0:1789760749.762174:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760749.762272:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508361856 00000100:00100000:0.0:1789760749.762282:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508361856 00000100:00100000:0.0:1789760749.762386:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728767329833 time 728822979828 state 0x0 ref 1 00000100:00100000:0.0:1789760749.762403:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728823240244 time 728822979828 state 0x1 ref 1 00000100:00100000:0.0:1789760749.762428:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728823240244 time 728867139372 state 0x0 ref 1 00000100:00100000:0.0:1789760749.762444:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508361856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760749.762493:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21757952, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:0.0:1789760749.762504:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5584f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.762524:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760749.762529:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760749.762537:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5584f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.762553:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5584f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.791881:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972632 is committed 00002000:00010000:2.0:1789760749.792215:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21757952 -> 21762048 00010000:00010000:2.0:1789760749.792232:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5584f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.792246:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5584f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.792255:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5584f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.792267:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.792272:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5584f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.792281:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5584f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.792444:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5584f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.792453:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5584f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21757952->21762047] (req 21757952->21762047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.792675:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508361856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30227us (30505us total) trans 4294972632 rc 0/0 00000100:00100000:3.0:1789760749.792686:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5344 00000100:00100000:3.0:1789760749.792697:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728823240244 time 728867139372 state 0x2 ref 0 00000100:00100000:0.0:1789760749.797706:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.798204:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508361984 00000100:00100000:2.0:1789760749.798216:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508361984 00000100:00100000:2.0:1789760749.798282:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728823240244 time 728867139372 state 0x0 ref 1 00000100:00100000:2.0:1789760749.798382:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728867399788 time 728867139372 state 0x1 ref 1 00000100:00100000:2.0:1789760749.798451:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728867399788 time 728903159966 state 0x0 ref 1 00000100:00100000:2.0:1789760749.798466:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508361984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.798520:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21762048, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:2.0:1789760749.798530:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55856 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.798553:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.798558:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.798565:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55856 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.798580:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55856 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760749.822789:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972633 is committed 00002000:00010000:3.0:1789760749.822793:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21762048 -> 21766144 00010000:00010000:3.0:1789760749.823042:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55856 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.823058:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55856 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.823067:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55856 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.823074:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760749.823079:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55856 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.823089:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55856 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.823114:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55856 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.823122:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55856 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21762048->21766143] (req 21762048->21766143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.823212:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508361984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24744us (25510us total) trans 4294972633 rc 0/0 00000100:00100000:3.0:1789760749.823228:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5345 00000100:00100000:3.0:1789760749.823238:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728867399788 time 728903159966 state 0x2 ref 0 00000100:00100000:0.0:1789760749.827651:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.828085:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508362112 00000100:00100000:2.0:1789760749.828096:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508362112 00000100:00100000:2.0:1789760749.828155:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728867399788 time 728903159966 state 0x0 ref 1 00000100:00100000:2.0:1789760749.828172:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728903420382 time 728903159966 state 0x1 ref 1 00000100:00100000:2.0:1789760749.828206:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728903420382 time 728932915624 state 0x0 ref 1 00000100:00100000:2.0:1789760749.828219:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508362112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.828272:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21766144, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:2.0:1789760749.828281:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5585d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.828334:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760749.828339:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.828351:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5585d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.828539:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5585d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.861631:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972634 is committed 00002000:00010000:3.0:1789760749.861754:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21766144 -> 21770240 00010000:00010000:3.0:1789760749.861772:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5585d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.861785:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5585d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.861805:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5585d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.861811:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760749.861815:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5585d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.861822:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5585d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.861841:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5585d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.861846:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5585d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21766144->21770239] (req 21766144->21770239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.861926:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508362112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33704us (34279us total) trans 4294972634 rc 0/0 00000100:00100000:3.0:1789760749.861937:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5346 00000100:00100000:3.0:1789760749.861948:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728903420382 time 728932915624 state 0x2 ref 0 00000100:00100000:2.0:1789760749.865724:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760749.865884:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508362240 00000100:00100000:3.0:1789760749.865894:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508362240 00000100:00100000:3.0:1789760749.865998:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728903420382 time 728932915624 state 0x0 ref 1 00000100:00100000:3.0:1789760749.866023:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728933176040 time 728932915624 state 0x1 ref 1 00000100:00100000:3.0:1789760749.866054:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728933176040 time 728970765454 state 0x0 ref 1 00000100:00100000:3.0:1789760749.866066:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508362240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760749.866120:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21770240, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:3.0:1789760749.866129:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55864 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.866145:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760749.866150:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.866156:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55864 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.866171:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55864 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760749.892159:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972635 is committed 00002000:00010000:0.0:1789760749.892524:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21770240 -> 21774336 00010000:00010000:0.0:1789760749.892540:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55864 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.892553:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55864 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.892562:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55864 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.892575:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760749.892580:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55864 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.892590:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55864 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.892611:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55864 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760749.892618:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55864 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21770240->21774335] (req 21770240->21774335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760749.892717:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508362240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26648us (26999us total) trans 4294972635 rc 0/0 00000100:00100000:0.0:1789760749.892727:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5347 00000100:00100000:0.0:1789760749.892737:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728933176040 time 728970765454 state 0x2 ref 0 00000100:00100000:0.0:1789760749.895978:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760749.896179:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508362368 00000100:00100000:3.0:1789760749.896190:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508362368 00000100:00100000:3.0:1789760749.896243:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728933176040 time 728970765454 state 0x0 ref 1 00000100:00100000:3.0:1789760749.896258:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728971025870 time 728970765454 state 0x1 ref 1 00000100:00100000:3.0:1789760749.896450:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728971025870 time 729001158267 state 0x0 ref 1 00000100:00100000:3.0:1789760749.896464:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508362368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760749.896517:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21774336, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:3.0:1789760749.896525:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5586b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.896543:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760749.896548:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.896556:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5586b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.896574:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5586b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.929322:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21774336 -> 21778432 00010000:00010000:3.0:1789760749.929342:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5586b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.929354:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5586b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.929483:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5586b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.929491:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760749.929497:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5586b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.929507:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5586b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.929532:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5586b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.929539:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5586b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21774336->21778431] (req 21774336->21778431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.929635:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508362368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33168us (33661us total) trans 4294972636 rc 0/0 00000100:00100000:3.0:1789760749.929646:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5348 00000100:00100000:3.0:1789760749.929656:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728971025870 time 729001158267 state 0x2 ref 0 00000001:00080000:3.0:1789760749.929697:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972636 is committed 00000100:00100000:3.0:1789760749.933684:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760749.934050:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508362496 00000100:00100000:2.0:1789760749.934062:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508362496 00000100:00100000:2.0:1789760749.934125:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 728971025870 time 729001158267 state 0x0 ref 1 00000100:00100000:2.0:1789760749.934143:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729001418683 time 729001158267 state 0x1 ref 1 00000100:00100000:2.0:1789760749.934178:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729001418683 time 729038888165 state 0x0 ref 1 00000100:00100000:2.0:1789760749.934192:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508362496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760749.934247:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21778432, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:2.0:1789760749.934257:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55872 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.934276:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760749.934281:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760749.934322:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55872 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.934339:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55872 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.963449:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972637 is committed 00002000:00010000:2.0:1789760749.963633:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21778432 -> 21782528 00010000:00010000:2.0:1789760749.963649:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55872 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.963662:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55872 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.963672:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55872 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.963679:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760749.963685:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55872 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.963694:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55872 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.963718:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55872 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760749.963726:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55872 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21778432->21782527] (req 21778432->21782527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760749.963879:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508362496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29681us (30196us total) trans 4294972637 rc 0/0 00000100:00100000:2.0:1789760749.963893:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5349 00000100:00100000:2.0:1789760749.963905:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729001418683 time 729038888165 state 0x2 ref 0 00000100:00100000:0.0:1789760749.968885:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760749.968977:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508362624 00000100:00100000:3.0:1789760749.968986:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508362624 00000100:00100000:3.0:1789760749.969052:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729001418683 time 729038888165 state 0x0 ref 1 00000100:00100000:3.0:1789760749.969067:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729039148581 time 729038888165 state 0x1 ref 1 00000100:00100000:3.0:1789760749.969102:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729039148581 time 729073812743 state 0x0 ref 1 00000100:00100000:3.0:1789760749.969117:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508362624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760749.969168:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21782528, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:3.0:1789760749.969178:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55879 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.969194:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760749.969199:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.969205:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55879 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.969220:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55879 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760750.006086:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21782528 -> 21786624 00010000:00010000:2.0:1789760750.006108:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55879 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.006122:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55879 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.006132:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55879 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.006140:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760750.006145:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55879 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.006155:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55879 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.006178:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972638 is committed 00010000:00010000:2.0:1789760750.006179:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55879 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.006186:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55879 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21782528->21786623] (req 21782528->21786623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.006347:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508362624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37167us (37407us total) trans 4294972638 rc 0/0 00000100:00100000:2.0:1789760750.006358:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5350 00000100:00100000:2.0:1789760750.006377:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729039148581 time 729073812743 state 0x2 ref 0 00000100:00100000:0.0:1789760750.011351:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760750.011436:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508362752 00000100:00100000:3.0:1789760750.011448:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508362752 00000100:00100000:3.0:1789760750.011507:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729039148581 time 729073812743 state 0x0 ref 1 00000100:00100000:3.0:1789760750.011525:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729074073159 time 729073812743 state 0x1 ref 1 00000100:00100000:3.0:1789760750.011554:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729074073159 time 729116264464 state 0x0 ref 1 00000100:00100000:3.0:1789760750.011569:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508362752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760750.011621:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21786624, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:3.0:1789760750.011631:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55880 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.011649:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760750.011654:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.011661:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55880 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.011677:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55880 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.048146:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972639 is committed 00002000:00010000:2.0:1789760750.048257:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21786624 -> 21790720 00010000:00010000:2.0:1789760750.048274:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55880 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.048307:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55880 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.048317:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55880 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.048325:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760750.048330:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55880 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.048340:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55880 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.048364:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55880 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.048378:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55880 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21786624->21790719] (req 21786624->21790719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.048475:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508362752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36903us (37128us total) trans 4294972639 rc 0/0 00000100:00100000:2.0:1789760750.048485:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5351 00000100:00100000:2.0:1789760750.048496:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729074073159 time 729116264464 state 0x2 ref 0 00000100:00100000:3.0:1789760750.058278:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760750.058893:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508362880 00000100:00100000:2.0:1789760750.058904:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508362880 00000100:00100000:2.0:1789760750.058964:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729074073159 time 729116264464 state 0x0 ref 1 00000100:00100000:2.0:1789760750.058982:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729116524880 time 729116264464 state 0x1 ref 1 00000100:00100000:2.0:1789760750.059004:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729116524880 time 729163715344 state 0x0 ref 1 00000100:00100000:2.0:1789760750.059032:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508362880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760750.059086:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21790720, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:2.0:1789760750.059096:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55887 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.059113:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760750.059117:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.059124:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55887 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.059146:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55887 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.100951:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972640 is committed 00002000:00010000:0.0:1789760750.101173:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21790720 -> 21794816 00010000:00010000:0.0:1789760750.101190:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55887 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.101208:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55887 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.101217:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55887 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.101225:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760750.101230:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55887 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.101240:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55887 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.101264:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55887 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.101272:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55887 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21790720->21794815] (req 21790720->21794815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.101599:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508362880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42564us (43326us total) trans 4294972640 rc 0/0 00000100:00100000:0.0:1789760750.101609:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5352 00000100:00100000:0.0:1789760750.101620:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729116524880 time 729163715344 state 0x2 ref 0 00000100:00100000:0.0:1789760750.105155:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760750.105245:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508363008 00000100:00100000:2.0:1789760750.105256:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508363008 00000100:00100000:2.0:1789760750.105569:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729116524880 time 729163715344 state 0x0 ref 1 00000100:00100000:2.0:1789760750.105585:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729163975760 time 729163715344 state 0x1 ref 1 00000100:00100000:2.0:1789760750.105611:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729163975760 time 729210321642 state 0x0 ref 1 00000100:00100000:2.0:1789760750.105624:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508363008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760750.105677:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21794816, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:2.0:1789760750.105687:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5588e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.105705:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760750.105710:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.105717:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5588e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.105732:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5588e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.153189:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972641 is committed 00002000:00010000:0.0:1789760750.153276:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21794816 -> 21798912 00010000:00010000:0.0:1789760750.153310:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5588e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.153323:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5588e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.153332:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5588e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.153339:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760750.153344:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5588e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.153354:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5588e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.153507:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5588e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.153516:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5588e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21794816->21798911] (req 21794816->21798911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.153648:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508363008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48021us (48498us total) trans 4294972641 rc 0/0 00000100:00100000:0.0:1789760750.153659:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5353 00000100:00100000:0.0:1789760750.153670:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729163975760 time 729210321642 state 0x2 ref 0 00000100:00100000:3.0:1789760750.157488:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760750.157639:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508363136 00000100:00100000:2.0:1789760750.157652:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508363136 00000100:00100000:2.0:1789760750.157709:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729163975760 time 729210321642 state 0x0 ref 1 00000100:00100000:2.0:1789760750.157725:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729210582058 time 729210321642 state 0x1 ref 1 00000100:00100000:2.0:1789760750.157751:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729210582058 time 729262461594 state 0x0 ref 1 00000100:00100000:2.0:1789760750.157764:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508363136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760750.157817:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21798912, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:2.0:1789760750.157827:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55895 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.157844:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760750.157849:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.157856:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55895 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.157872:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55895 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.206841:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972642 is committed 00002000:00010000:3.0:1789760750.207003:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21798912 -> 21803008 00010000:00010000:3.0:1789760750.207048:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55895 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.207061:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55895 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.207069:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55895 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.207076:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760750.207081:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55895 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.207095:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55895 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.207118:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55895 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.207126:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55895 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21798912->21803007] (req 21798912->21803007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.207226:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508363136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49458us (49742us total) trans 4294972642 rc 0/0 00000100:00100000:3.0:1789760750.207237:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5354 00000100:00100000:3.0:1789760750.207248:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729210582058 time 729262461594 state 0x2 ref 0 00000100:00100000:3.0:1789760750.210757:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760750.210845:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508363264 00000100:00100000:0.0:1789760750.210855:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508363264 00000100:00100000:0.0:1789760750.211031:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729210582058 time 729262461594 state 0x0 ref 1 00000100:00100000:0.0:1789760750.211048:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729262722010 time 729262461594 state 0x1 ref 1 00000100:00100000:0.0:1789760750.211083:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729262722010 time 729315792418 state 0x0 ref 1 00000100:00100000:0.0:1789760750.211097:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508363264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760750.211147:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21803008, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:0.0:1789760750.211157:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5589c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.211176:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760750.211181:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.211188:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5589c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.211203:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5589c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.242770:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972643 is committed 00002000:00010000:2.0:1789760750.242877:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21803008 -> 21807104 00010000:00010000:2.0:1789760750.242895:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5589c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.243209:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5589c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.243229:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5589c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.243237:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760750.243242:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5589c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.243251:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5589c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.243276:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5589c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.243304:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5589c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21803008->21807103] (req 21803008->21807103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.243517:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508363264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32417us (32762us total) trans 4294972643 rc 0/0 00000100:00100000:2.0:1789760750.243528:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5355 00000100:00100000:2.0:1789760750.243539:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729262722010 time 729315792418 state 0x2 ref 0 00000100:00100000:0.0:1789760750.249140:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760750.249264:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508363392 00000100:00100000:2.0:1789760750.249276:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508363392 00000100:00100000:2.0:1789760750.249355:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729262722010 time 729315792418 state 0x0 ref 1 00000100:00100000:2.0:1789760750.249436:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729316052834 time 729315792418 state 0x1 ref 1 00000100:00100000:2.0:1789760750.249507:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729316052834 time 729354216944 state 0x0 ref 1 00000100:00100000:2.0:1789760750.249521:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508363392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760750.249572:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21807104, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:2.0:1789760750.249582:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe558a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.249600:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760750.249605:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.249613:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe558a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.249629:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe558a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.281699:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972644 is committed 00002000:00010000:0.0:1789760750.281798:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21807104 -> 21811200 00010000:00010000:0.0:1789760750.281814:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe558a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.281833:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe558a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.281842:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe558a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.281848:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760750.281853:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe558a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.281863:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe558a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.281884:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe558a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.281892:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe558a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21807104->21811199] (req 21807104->21811199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.282077:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508363392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32553us (32943us total) trans 4294972644 rc 0/0 00000100:00100000:0.0:1789760750.282087:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5356 00000100:00100000:0.0:1789760750.282098:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729316052834 time 729354216944 state 0x2 ref 0 00000100:00100000:0.0:1789760750.285356:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760750.285615:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508363520 00000100:00100000:3.0:1789760750.285626:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508363520 00000100:00100000:3.0:1789760750.285689:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729316052834 time 729354216944 state 0x0 ref 1 00000100:00100000:3.0:1789760750.285706:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729354477360 time 729354216944 state 0x1 ref 1 00000100:00100000:3.0:1789760750.285768:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729354477360 time 729390478157 state 0x0 ref 1 00000100:00100000:3.0:1789760750.285782:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508363520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760750.285834:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21811200, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:3.0:1789760750.285845:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.285862:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760750.285866:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.285872:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.285887:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.313529:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972645 is committed 00002000:00010000:3.0:1789760750.313655:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21811200 -> 21815296 00010000:00010000:3.0:1789760750.313671:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.313685:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.313694:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.313702:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760750.313707:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.313717:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.313742:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.313750:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21811200->21815295] (req 21811200->21815295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.313915:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508363520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28130us (28564us total) trans 4294972645 rc 0/0 00000100:00100000:3.0:1789760750.313932:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5357 00000100:00100000:3.0:1789760750.313945:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729354477360 time 729390478157 state 0x2 ref 0 00000100:00100000:0.0:1789760750.318023:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760750.318105:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508363648 00000100:00100000:2.0:1789760750.318116:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508363648 00000100:00100000:2.0:1789760750.318173:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729354477360 time 729390478157 state 0x0 ref 1 00000100:00100000:2.0:1789760750.318191:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729390738573 time 729390478157 state 0x1 ref 1 00000100:00100000:2.0:1789760750.318259:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729390738573 time 729422968356 state 0x0 ref 1 00000100:00100000:2.0:1789760750.318272:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508363648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760750.318367:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21815296, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:2.0:1789760750.318378:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.318396:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760750.318401:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.318408:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.318425:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.334463:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760750.346377:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972646 is committed 00002000:00010000:2.0:1789760750.346531:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21815296 -> 21819392 00010000:00010000:2.0:1789760750.346548:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.346561:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.346569:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.346576:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760750.346581:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.346591:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.346616:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.346624:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21815296->21819391] (req 21815296->21819391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.346723:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508363648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28447us (28713us total) trans 4294972646 rc 0/0 00000100:00100000:2.0:1789760750.346733:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5358 00000100:00100000:2.0:1789760750.346743:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729390738573 time 729422968356 state 0x2 ref 0 00000100:00100000:0.0:1789760750.350735:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760750.351095:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508363776 00000100:00100000:1.0:1789760750.351106:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508363776 00000100:00100000:1.0:1789760750.351164:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729390738573 time 729422968356 state 0x0 ref 1 00000100:00100000:1.0:1789760750.351182:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729423228772 time 729422968356 state 0x1 ref 1 00000100:00100000:1.0:1789760750.351213:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729423228772 time 729455923139 state 0x0 ref 1 00000100:00100000:1.0:1789760750.351244:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508363776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760750.351313:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21819392, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:1.0:1789760750.351324:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe558b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.351341:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760750.351346:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.351353:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe558b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.351377:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe558b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.384245:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972647 is committed 00002000:00010000:1.0:1789760750.384349:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21819392 -> 21823488 00010000:00010000:1.0:1789760750.384428:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe558b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.384441:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe558b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.384449:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe558b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.384456:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760750.384461:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe558b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.384471:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe558b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.384493:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe558b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.384500:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe558b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21819392->21823487] (req 21819392->21823487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.384622:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508363776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33376us (33892us total) trans 4294972647 rc 0/0 00000100:00100000:1.0:1789760750.384632:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5359 00000100:00100000:1.0:1789760750.384642:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729423228772 time 729455923139 state 0x2 ref 0 00000100:00100000:0.0:1789760750.388991:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760750.389072:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508363904 00000100:00100000:3.0:1789760750.389082:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508363904 00000100:00100000:3.0:1789760750.389138:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729423228772 time 729455923139 state 0x0 ref 1 00000100:00100000:3.0:1789760750.389155:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729456183555 time 729455923139 state 0x1 ref 1 00000100:00100000:3.0:1789760750.389217:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729456183555 time 729493926742 state 0x0 ref 1 00000100:00100000:3.0:1789760750.389230:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508363904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760750.389281:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21823488, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:3.0:1789760750.389325:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.389347:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760750.389352:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.389369:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.389386:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.426204:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972648 is committed 00002000:00010000:1.0:1789760750.426565:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21823488 -> 21827584 00010000:00010000:1.0:1789760750.426581:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.426594:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.426603:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.426611:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760750.426617:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.426626:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.426648:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.426655:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe558bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21823488->21827583] (req 21823488->21827583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760750.426744:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508363904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37510us (37758us total) trans 4294972648 rc 0/0 00000100:00100000:1.0:1789760750.426753:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5360 00000100:00100000:1.0:1789760750.426764:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729456183555 time 729493926742 state 0x2 ref 0 00000100:00100000:1.0:1789760750.430719:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760750.430810:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508364032 00000100:00100000:2.0:1789760750.430821:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508364032 00000100:00100000:2.0:1789760750.430876:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729456183555 time 729493926742 state 0x0 ref 1 00000100:00100000:2.0:1789760750.430891:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729494187158 time 729493926742 state 0x1 ref 1 00000100:00100000:2.0:1789760750.431031:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729494187158 time 729535739855 state 0x0 ref 1 00000100:00100000:2.0:1789760750.431045:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508364032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760750.431096:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21827584, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:2.0:1789760750.431107:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.431129:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760750.431133:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.431141:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.431156:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.459653:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972649 is committed 00002000:00010000:2.0:1789760750.459747:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21827584 -> 21831680 00010000:00010000:2.0:1789760750.459763:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.459777:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.459787:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.459794:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760750.459799:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.459809:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.459834:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.459843:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21827584->21831679] (req 21827584->21831679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.460013:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508364032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28957us (29290us total) trans 4294972649 rc 0/0 00000100:00100000:2.0:1789760750.460024:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5361 00000100:00100000:2.0:1789760750.460035:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729494187158 time 729535739855 state 0x2 ref 0 00000100:00100000:1.0:1789760750.466581:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760750.467219:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508364160 00000100:00100000:0.0:1789760750.467231:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508364160 00000100:00100000:0.0:1789760750.467324:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729494187158 time 729535739855 state 0x0 ref 1 00000100:00100000:0.0:1789760750.467341:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729536000271 time 729535739855 state 0x1 ref 1 00000100:00100000:0.0:1789760750.467377:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729536000271 time 729572087183 state 0x0 ref 1 00000100:00100000:0.0:1789760750.467389:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508364160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760750.467437:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21831680, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:0.0:1789760750.467448:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe558cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.467465:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760750.467470:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.467477:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe558cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.467492:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe558cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.532112:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972650 is committed 00002000:00010000:3.0:1789760750.532274:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21831680 -> 21835776 00010000:00010000:3.0:1789760750.532328:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe558cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.532342:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe558cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.532351:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe558cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.532368:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760750.532373:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe558cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.532383:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe558cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.532411:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe558cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.532419:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe558cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21831680->21835775] (req 21831680->21835775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.532505:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508364160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 65113us (65926us total) trans 4294972650 rc 0/0 00000100:00100000:3.0:1789760750.532516:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5362 00000100:00100000:3.0:1789760750.532526:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729536000271 time 729572087183 state 0x2 ref 0 00000100:00100000:1.0:1789760750.538598:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760750.538678:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508364288 00000100:00100000:3.0:1789760750.538687:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508364288 00000100:00100000:3.0:1789760750.538746:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729536000271 time 729572087183 state 0x0 ref 1 00000100:00100000:3.0:1789760750.538762:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729572347599 time 729572087183 state 0x1 ref 1 00000100:00100000:3.0:1789760750.538830:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729572347599 time 729643539372 state 0x0 ref 1 00000100:00100000:3.0:1789760750.538843:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508364288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760750.538894:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21835776, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0xa807 00010000:00010000:3.0:1789760750.538903:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.538929:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760750.538934:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.538942:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.538957:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760750.570344:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972651 is committed 00002000:00010000:3.0:1789760750.570640:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21835776 -> 21839872 00002000:00010000:3.0:1789760750.570645:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760750 -> 1789760751 00002000:00010000:3.0:1789760750.570648:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760750 -> 1789760751 00010000:00010000:3.0:1789760750.570662:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.570676:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.570686:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.570693:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760750.570698:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.570707:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.570730:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.570739:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21835776->21839871] (req 21835776->21839871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.570878:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508364288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32030us (32283us total) trans 4294972651 rc 0/0 00000100:00100000:3.0:1789760750.570889:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5363 00000100:00100000:3.0:1789760750.570899:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729572347599 time 729643539372 state 0x2 ref 0 00000100:00100000:0.0:1789760750.577626:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760750.578206:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508364416 00000100:00100000:2.0:1789760750.578218:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508364416 00000100:00100000:2.0:1789760750.578274:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729572347599 time 729643539372 state 0x0 ref 1 00000100:00100000:2.0:1789760750.578320:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729643799788 time 729643539372 state 0x1 ref 1 00000100:00100000:2.0:1789760750.578351:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729643799788 time 729683061061 state 0x0 ref 1 00000100:00100000:2.0:1789760750.578473:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508364416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760750.578527:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21839872, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:2.0:1789760750.578538:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.578558:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760750.578563:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.578571:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.578589:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.632566:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972652 is committed 00002000:00010000:2.0:1789760750.632655:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21839872 -> 21843968 00010000:00010000:2.0:1789760750.632671:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.632684:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.632693:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.632700:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760750.632704:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.632714:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.632735:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.632743:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe558db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21839872->21843967] (req 21839872->21843967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.632838:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508364416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54366us (55218us total) trans 4294972652 rc 0/0 00000100:00100000:2.0:1789760750.632848:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5364 00000100:00100000:2.0:1789760750.632858:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729643799788 time 729683061061 state 0x2 ref 0 00000100:00100000:2.0:1789760750.637198:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760750.637282:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508364544 00000100:00100000:3.0:1789760750.637325:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508364544 00000100:00100000:3.0:1789760750.637390:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729643799788 time 729683061061 state 0x0 ref 1 00000100:00100000:3.0:1789760750.637405:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729683321477 time 729683061061 state 0x1 ref 1 00000100:00100000:3.0:1789760750.637429:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729683321477 time 729742140728 state 0x0 ref 1 00000100:00100000:3.0:1789760750.637442:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508364544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760750.637494:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21843968, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:3.0:1789760750.637503:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.637522:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760750.637526:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.637534:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.637550:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.656451:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760750.656533:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760750.656563:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760750.656798:0:6459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508364672 00000100:00100000:1.0:1789760750.656809:0:6459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508364672 00000100:00100000:1.0:1789760750.656862:0:6459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 475 00000100:00100000:1.0:1789760750.656872:0:6459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988caec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2407:x1876699508364672:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760750.656937:0:6459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988caec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2407:x1876699508364672:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 66us (403us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760750.656944:0:6459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 475 00000100:00100000:3.0:1789760750.661411:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760750.664508:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508364800 00000100:00100000:2.0:1789760750.664524:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508364800 00000100:00100000:2.0:1789760750.664618:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508364928 00000100:00100000:2.0:1789760750.664623:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508364928 00000100:00100000:2.0:1789760750.664664:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 909 00000100:00100000:2.0:1789760750.664675:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2406:x1876699508364800:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760750.664759:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2406:x1876699508364800:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 86us (8316us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760750.664768:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 909 00000100:00100000:2.0:1789760750.664786:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 910 00000100:00100000:2.0:1789760750.664793:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508364928:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760750.664821:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508364928:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 29us (8257us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760750.664828:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 910 00000100:00080000:0.0:1789760750.666404:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789760750.666413:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760750.669022:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508365184 00000100:00100000:1.0:1789760750.669037:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508365184 00000100:00100000:1.0:1789760750.669113:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 911 00000100:00100000:1.0:1789760750.669123:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508365184:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760750.669205:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508365184:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 83us (7802us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760750.669213:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 911 00000100:00100000:1.0:1789760750.669277:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760750.669421:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508365056 00000100:00100000:1.0:1789760750.669425:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508365056 00000100:00100000:1.0:1789760750.669452:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 912 00000100:00100000:1.0:1789760750.669459:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2406:x1876699508365056:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760750.669493:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2406:x1876699508365056:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 34us (216us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760750.669499:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 912 00000100:00080000:2.0:1789760750.669941:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760750.669947:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00002000:00010000:3.0:1789760750.673066:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21843968 -> 21848064 00010000:00010000:3.0:1789760750.673159:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.673173:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.673182:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.673190:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760750.673194:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.673208:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.673232:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.673241:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21843968->21848063] (req 21843968->21848063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.673532:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508364544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36086us (36338us total) trans 4294972653 rc 0/0 00000100:00100000:3.0:1789760750.673543:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5365 00000100:00100000:3.0:1789760750.673554:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729683321477 time 729742140728 state 0x2 ref 0 00000001:00080000:2.0:1789760750.674193:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972653 is committed 00000100:00100000:1.0:1789760750.681663:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760750.681754:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508365312 00000100:00100000:3.0:1789760750.681766:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508365312 00000100:00100000:3.0:1789760750.681827:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729683321477 time 729742140728 state 0x0 ref 1 00000100:00100000:3.0:1789760750.681844:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729742401144 time 729742140728 state 0x1 ref 1 00000100:00100000:3.0:1789760750.681905:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729742401144 time 729786614214 state 0x0 ref 1 00000100:00100000:3.0:1789760750.681918:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508365312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760750.681971:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21848064, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:3.0:1789760750.681982:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.682001:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760750.682016:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.682024:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.682046:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.713530:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972654 is committed 00002000:00010000:3.0:1789760750.713679:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21848064 -> 21852160 00010000:00010000:3.0:1789760750.713695:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.713706:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.713714:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.713720:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760750.713726:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.713734:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.713753:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.713759:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe558e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21848064->21852159] (req 21848064->21852159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.713855:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508365312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31933us (32197us total) trans 4294972654 rc 0/0 00000100:00100000:3.0:1789760750.713864:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5366 00000100:00100000:3.0:1789760750.713875:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729742401144 time 729786614214 state 0x2 ref 0 00000100:00100000:1.0:1789760750.717814:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760750.717898:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508365440 00000100:00100000:2.0:1789760750.717909:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508365440 00000100:00100000:2.0:1789760750.717968:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729742401144 time 729786614214 state 0x0 ref 1 00000100:00100000:2.0:1789760750.717984:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729786874630 time 729786614214 state 0x1 ref 1 00000100:00100000:2.0:1789760750.718037:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729786874630 time 729822747695 state 0x0 ref 1 00000100:00100000:2.0:1789760750.718051:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508365440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760750.718101:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21852160, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:2.0:1789760750.718111:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe558f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.718129:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760750.718134:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.718143:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe558f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.718160:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe558f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760750.752834:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21852160 -> 21856256 00010000:00010000:0.0:1789760750.752856:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe558f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.752870:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe558f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.752880:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe558f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.752887:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760750.752893:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe558f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.752902:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe558f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.752926:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe558f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.752934:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe558f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21852160->21856255] (req 21852160->21856255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760750.753045:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508365440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34991us (35235us total) trans 4294972655 rc 0/0 00000100:00100000:0.0:1789760750.753056:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5367 00000100:00100000:0.0:1789760750.753067:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729786874630 time 729822747695 state 0x2 ref 0 00000001:00080000:1.0:1789760750.753952:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972655 is committed 00000100:00100000:1.0:1789760750.763391:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760750.763498:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508365568 00000100:00100000:3.0:1789760750.763509:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508365568 00000100:00100000:3.0:1789760750.763568:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729786874630 time 729822747695 state 0x0 ref 1 00000100:00100000:3.0:1789760750.763585:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729823008111 time 729822747695 state 0x1 ref 1 00000100:00100000:3.0:1789760750.763618:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729823008111 time 729868327925 state 0x0 ref 1 00000100:00100000:3.0:1789760750.763632:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508365568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760750.763684:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21856256, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:3.0:1789760750.763695:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe558f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.763713:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760750.763718:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.763724:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe558f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.763750:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe558f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760750.814840:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972656 is committed 00002000:00010000:2.0:1789760750.815013:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21856256 -> 21860352 00010000:00010000:2.0:1789760750.815029:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe558f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.815042:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe558f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.815052:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe558f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.815060:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760750.815065:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe558f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.815074:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe558f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.815098:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe558f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.815105:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe558f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21856256->21860351] (req 21856256->21860351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.815200:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508365568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51565us (51812us total) trans 4294972656 rc 0/0 00000100:00100000:2.0:1789760750.815210:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5368 00000100:00100000:2.0:1789760750.815220:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729823008111 time 729868327925 state 0x2 ref 0 00000100:00100000:3.0:1789760750.818500:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760750.818786:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508365696 00000100:00100000:1.0:1789760750.818797:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508365696 00000100:00100000:1.0:1789760750.818857:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729823008111 time 729868327925 state 0x0 ref 1 00000100:00100000:1.0:1789760750.818873:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729868588341 time 729868327925 state 0x1 ref 1 00000100:00100000:1.0:1789760750.819660:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729868588341 time 729924365179 state 0x0 ref 1 00000100:00100000:1.0:1789760750.819695:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508365696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760750.820255:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21860352, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:1.0:1789760750.820268:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe558fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.820301:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760750.820306:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760750.820315:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe558fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760750.820331:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe558fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.867526:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21860352 -> 21864448 00010000:00010000:3.0:1789760750.867546:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe558fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.867559:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe558fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.867569:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe558fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.867577:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760750.867582:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe558fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.867592:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe558fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.867615:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe558fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.867623:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe558fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21860352->21864447] (req 21860352->21864447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.867720:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508365696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48030us (49365us total) trans 4294972657 rc 0/0 00000100:00100000:3.0:1789760750.867730:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5369 00000100:00100000:3.0:1789760750.867740:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729868588341 time 729924365179 state 0x2 ref 0 00000001:00080000:0.0:1789760750.868022:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972657 is committed 00000100:00100000:1.0:1789760750.872839:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760750.872933:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508365824 00000100:00100000:0.0:1789760750.872944:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508365824 00000100:00100000:0.0:1789760750.873017:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729868588341 time 729924365179 state 0x0 ref 1 00000100:00100000:0.0:1789760750.873034:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729924625595 time 729924365179 state 0x1 ref 1 00000100:00100000:0.0:1789760750.873068:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729924625595 time 729977778222 state 0x0 ref 1 00000100:00100000:0.0:1789760750.873082:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508365824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760750.873146:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21864448, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:0.0:1789760750.873156:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55905 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.873177:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760750.873182:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760750.873190:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55905 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760750.873204:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55905 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.915382:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972658 is committed 00002000:00010000:3.0:1789760750.915558:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21864448 -> 21868544 00010000:00010000:3.0:1789760750.915575:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55905 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.915587:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55905 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.915597:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55905 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.915604:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760750.915609:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55905 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.915618:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55905 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.915643:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55905 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.915652:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55905 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21864448->21868543] (req 21864448->21868543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.915840:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508365824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42755us (43007us total) trans 4294972658 rc 0/0 00000100:00100000:3.0:1789760750.915852:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5370 00000100:00100000:3.0:1789760750.915863:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729924625595 time 729977778222 state 0x2 ref 0 00000100:00100000:2.0:1789760750.919828:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760750.919928:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508365952 00000100:00100000:3.0:1789760750.919938:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508365952 00000100:00100000:3.0:1789760750.919998:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729924625595 time 729977778222 state 0x0 ref 1 00000100:00100000:3.0:1789760750.920026:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729978038638 time 729977778222 state 0x1 ref 1 00000100:00100000:3.0:1789760750.920058:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729978038638 time 730024768603 state 0x0 ref 1 00000100:00100000:3.0:1789760750.920072:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508365952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760750.920131:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21868544, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:3.0:1789760750.920141:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5590c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.920159:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760750.920163:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.920170:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5590c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.920185:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5590c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760750.963228:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972659 is committed 00002000:00010000:3.0:1789760750.963748:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21868544 -> 21872640 00010000:00010000:3.0:1789760750.963765:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5590c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.963778:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5590c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.963788:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5590c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.963795:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760750.963801:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5590c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.963810:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5590c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.963834:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5590c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.963843:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5590c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21868544->21872639] (req 21868544->21872639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.963977:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508365952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43901us (44156us total) trans 4294972659 rc 0/0 00000100:00100000:3.0:1789760750.963992:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5371 00000100:00100000:3.0:1789760750.964013:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729978038638 time 730024768603 state 0x2 ref 0 00000100:00100000:0.0:1789760750.968619:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760750.969191:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508366080 00000100:00100000:3.0:1789760750.969203:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508366080 00000100:00100000:3.0:1789760750.969885:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 729978038638 time 730024768603 state 0x0 ref 1 00000100:00100000:3.0:1789760750.969903:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730025029019 time 730024768603 state 0x1 ref 1 00000100:00100000:2.0:1789760750.970117:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730025029019 time 730074816544 state 0x0 ref 1 00000100:00100000:2.0:1789760750.970137:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508366080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760750.970190:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21872640, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:2.0:1789760750.970200:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55913 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.970218:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760750.970223:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760750.970242:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55913 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.970259:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55913 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.998930:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972660 is committed 00002000:00010000:2.0:1789760750.999204:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21872640 -> 21876736 00010000:00010000:2.0:1789760750.999220:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55913 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.999233:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55913 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.999243:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55913 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.999251:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760750.999257:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55913 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.999266:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55913 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.999312:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55913 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760750.999321:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55913 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21872640->21876735] (req 21872640->21876735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760750.999516:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508366080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29379us (30900us total) trans 4294972660 rc 0/0 00000100:00100000:2.0:1789760750.999527:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5372 00000100:00100000:2.0:1789760750.999538:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730025029019 time 730074816544 state 0x2 ref 0 00000100:00100000:1.0:1789760751.010754:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760751.010851:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508366208 00000100:00100000:3.0:1789760751.010863:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508366208 00000100:00100000:3.0:1789760751.010939:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730025029019 time 730074816544 state 0x0 ref 1 00000100:00100000:3.0:1789760751.010957:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730075076960 time 730074816544 state 0x1 ref 1 00000100:00100000:3.0:1789760751.010990:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730075076960 time 730115700267 state 0x0 ref 1 00000100:00100000:3.0:1789760751.011018:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508366208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760751.011069:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21876736, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:3.0:1789760751.011079:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5591a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.011097:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.011102:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.011109:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5591a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.011144:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5591a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760751.011388:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760751.059841:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972661 is committed 00002000:00010000:3.0:1789760751.062000:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21876736 -> 21880832 00010000:00010000:3.0:1789760751.062031:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5591a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.062046:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5591a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.062055:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5591a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.062062:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760751.062067:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5591a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.062076:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5591a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.062100:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5591a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.062109:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5591a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21876736->21880831] (req 21876736->21880831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.062230:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508366208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51222us (51480us total) trans 4294972661 rc 0/0 00000100:00100000:3.0:1789760751.062242:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5373 00000100:00100000:3.0:1789760751.062255:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730075076960 time 730115700267 state 0x2 ref 0 00000100:00100000:1.0:1789760751.065771:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760751.065897:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508366336 00000100:00100000:3.0:1789760751.065908:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508366336 00000100:00100000:3.0:1789760751.065968:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730075076960 time 730115700267 state 0x0 ref 1 00000100:00100000:3.0:1789760751.065986:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730115960683 time 730115700267 state 0x1 ref 1 00000100:00100000:3.0:1789760751.066018:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730115960683 time 730170728372 state 0x0 ref 1 00000100:00100000:3.0:1789760751.066030:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508366336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760751.066077:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21880832, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:3.0:1789760751.066086:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55921 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.066103:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.066108:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.066115:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55921 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.066131:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55921 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.119812:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972662 is committed 00002000:00010000:0.0:1789760751.120092:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21880832 -> 21884928 00010000:00010000:0.0:1789760751.120109:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55921 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.120121:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55921 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.120130:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55921 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.120137:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760751.120142:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55921 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.120151:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55921 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.120176:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55921 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.120184:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55921 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21880832->21884927] (req 21880832->21884927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.120275:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508366336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54242us (54512us total) trans 4294972662 rc 0/0 00000100:00100000:0.0:1789760751.121563:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5374 00000100:00100000:0.0:1789760751.121581:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730115960683 time 730170728372 state 0x2 ref 0 00000100:00100000:0.0:1789760751.127332:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760751.127620:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508366464 00000100:00100000:2.0:1789760751.127631:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508366464 00000100:00100000:2.0:1789760751.127688:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730115960683 time 730170728372 state 0x0 ref 1 00000100:00100000:2.0:1789760751.127711:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730170988788 time 730170728372 state 0x1 ref 1 00000100:00100000:2.0:1789760751.127769:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730170988788 time 730232478640 state 0x0 ref 1 00000100:00100000:2.0:1789760751.127782:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508366464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760751.127831:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21884928, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:2.0:1789760751.127842:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55928 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.127860:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760751.127865:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.127872:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55928 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.127889:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55928 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.187683:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972663 is committed 00002000:00010000:3.0:1789760751.187968:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21884928 -> 21889024 00010000:00010000:3.0:1789760751.187985:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55928 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.187997:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55928 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.188079:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55928 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.188089:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760751.188094:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55928 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.188103:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55928 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.188128:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55928 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.188135:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe55928 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21884928->21889023] (req 21884928->21889023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.188233:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508366464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60448us (60905us total) trans 4294972663 rc 0/0 00000100:00100000:3.0:1789760751.188245:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5375 00000100:00100000:3.0:1789760751.188258:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730170988788 time 730232478640 state 0x2 ref 0 00000100:00100000:2.0:1789760751.201997:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760751.202887:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508366592 00000100:00100000:3.0:1789760751.202900:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508366592 00000100:00100000:3.0:1789760751.202963:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730170988788 time 730232478640 state 0x0 ref 1 00000100:00100000:3.0:1789760751.202979:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730232739056 time 730232478640 state 0x1 ref 1 00000100:00100000:3.0:1789760751.203068:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730232739056 time 730307777319 state 0x0 ref 1 00000100:00100000:3.0:1789760751.203083:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508366592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760751.203134:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21889024, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:3.0:1789760751.203146:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5592f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.203164:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.203169:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.203177:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5592f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.203193:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5592f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.264276:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972664 is committed 00002000:00010000:3.0:1789760751.264504:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21889024 -> 21893120 00010000:00010000:3.0:1789760751.264522:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5592f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.264537:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5592f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.264547:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5592f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.264555:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760751.264560:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5592f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.264570:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5592f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.264594:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5592f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.264603:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5592f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21889024->21893119] (req 21889024->21893119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.264701:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508366592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61616us (62708us total) trans 4294972664 rc 0/0 00000100:00100000:3.0:1789760751.264711:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5376 00000100:00100000:3.0:1789760751.264731:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730232739056 time 730307777319 state 0x2 ref 0 00000100:00100000:1.0:1789760751.276083:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760751.276244:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508366720 00000100:00100000:2.0:1789760751.276256:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508366720 00000100:00100000:2.0:1789760751.276353:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730232739056 time 730307777319 state 0x0 ref 1 00000100:00100000:2.0:1789760751.276379:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730308037735 time 730307777319 state 0x1 ref 1 00000100:00100000:2.0:1789760751.276402:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730308037735 time 730381112592 state 0x0 ref 1 00000100:00100000:2.0:1789760751.276415:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508366720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760751.276469:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21893120, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:2.0:1789760751.276479:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55936 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.276497:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760751.276502:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.276509:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55936 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.276525:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55936 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.355461:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972665 is committed 00002000:00010000:3.0:1789760751.355611:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21893120 -> 21897216 00010000:00010000:3.0:1789760751.355629:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55936 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.355642:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55936 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.355651:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55936 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.355658:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760751.355663:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55936 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.355672:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55936 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.355695:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55936 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.355703:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe55936 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21893120->21897215] (req 21893120->21897215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.355865:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508366720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 79447us (79789us total) trans 4294972665 rc 0/0 00000100:00100000:3.0:1789760751.355876:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5377 00000100:00100000:3.0:1789760751.355886:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730308037735 time 730381112592 state 0x2 ref 0 00010000:00010000:3.0:1789760751.358435:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760751.360047:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760751.360143:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508366848 00000100:00100000:3.0:1789760751.360153:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508366848 00000100:00100000:3.0:1789760751.360212:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730308037735 time 730381112592 state 0x0 ref 1 00000100:00100000:3.0:1789760751.360229:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730381373008 time 730381112592 state 0x1 ref 1 00000100:00100000:3.0:1789760751.360259:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730381373008 time 730464969772 state 0x0 ref 1 00000100:00100000:3.0:1789760751.360270:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508366848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760751.363194:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21897216, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:3.0:1789760751.363210:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5593d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.363229:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.363233:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.363240:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5593d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.363256:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5593d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.398607:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972666 is committed 00002000:00010000:3.0:1789760751.398780:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21897216 -> 21901312 00010000:00010000:3.0:1789760751.398796:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5593d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.398815:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5593d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.398824:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5593d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.398831:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760751.398836:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5593d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.398845:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5593d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.398869:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5593d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.398876:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5593d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21897216->21901311] (req 21897216->21901311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.398986:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508366848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38711us (39068us total) trans 4294972666 rc 0/0 00000100:00100000:3.0:1789760751.398997:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5378 00000100:00100000:3.0:1789760751.399016:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730381373008 time 730464969772 state 0x2 ref 0 00000100:00100000:3.0:1789760751.403387:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760751.403577:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508366976 00000100:00100000:0.0:1789760751.403590:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508366976 00000100:00100000:0.0:1789760751.403647:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730381373008 time 730464969772 state 0x0 ref 1 00000100:00100000:0.0:1789760751.403664:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730465230188 time 730464969772 state 0x1 ref 1 00000100:00100000:0.0:1789760751.403698:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730465230188 time 730508408610 state 0x0 ref 1 00000100:00100000:0.0:1789760751.403711:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508366976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760751.403774:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21901312, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:0.0:1789760751.403784:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55944 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.403800:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760751.403805:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.403811:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55944 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.403826:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55944 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760751.462266:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21901312 -> 21905408 00010000:00010000:0.0:1789760751.462303:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55944 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.462322:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55944 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.462331:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55944 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.462339:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760751.462344:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55944 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.462363:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55944 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.462385:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55944 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.462393:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55944 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21901312->21905407] (req 21901312->21905407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.462477:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508366976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58762us (59094us total) trans 4294972667 rc 0/0 00000100:00100000:0.0:1789760751.462487:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5379 00000100:00100000:0.0:1789760751.462497:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730465230188 time 730508408610 state 0x2 ref 0 00000001:00080000:0.0:1789760751.462532:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972667 is committed 00000100:00100000:3.0:1789760751.465653:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760751.465693:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508367104 00000100:00100000:3.0:1789760751.465702:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508367104 00000100:00100000:3.0:1789760751.465759:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730465230188 time 730508408610 state 0x0 ref 1 00000100:00100000:3.0:1789760751.465777:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730508669026 time 730508408610 state 0x1 ref 1 00000100:00100000:3.0:1789760751.465803:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730508669026 time 730570513874 state 0x0 ref 1 00000100:00100000:3.0:1789760751.465816:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508367104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760751.465870:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21905408, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0xa807 00010000:00010000:3.0:1789760751.465879:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5594b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.465897:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.465902:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.465909:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5594b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.465937:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5594b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.490523:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7efa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534073216:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760751.490594:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.490652:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760751.490659:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04886363100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534073600:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760751.490684:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.490714:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7ef8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534073344:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760751.490739:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.490757:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789760751.490762:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7efaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534073472:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760751.490785:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.490816:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534073216 00000100:00100000:2.0:1789760751.490823:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534073216 00000100:00100000:2.0:1789760751.490871:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534073600 00000100:00100000:2.0:1789760751.490874:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534073600 00000100:00100000:2.0:1789760751.490886:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534073344 00000100:00100000:2.0:1789760751.490889:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534073344 00000100:00100000:2.0:1789760751.490902:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534073472 00000100:00100000:2.0:1789760751.490905:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534073472 00000100:00100000:2.0:1789760751.490938:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:2.0:1789760751.490947:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534073216:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760751.490972:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527499264 pending 1703936 free 3836739584 avail 3825205248 00000100:00100000:2.0:1789760751.491030:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534073216:12345-0@lo:13:osp-pre-0-1.0 Request processed in 83us (434us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.491037:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00000100:00100000:2.0:1789760751.491053:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 377 00000100:00100000:2.0:1789760751.491059:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534073600:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760751.491082:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760751.491108:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534073600:12345-0@lo:13:osp-pre-1-0.0 Request processed in 49us (424us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.491115:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 377 00000100:00100000:2.0:1789760751.491131:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 378 00000100:00100000:2.0:1789760751.491137:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534073344:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760751.491146:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527499264 pending 1703936 free 3836739584 avail 3825205248 00000100:00100000:2.0:1789760751.491173:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534073344:12345-0@lo:13:osp-pre-0-0.0 Request processed in 36us (433us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.491178:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 378 00000100:00100000:2.0:1789760751.491189:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 379 00000100:00100000:2.0:1789760751.491194:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534073472:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789760751.491203:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760751.491222:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534073472:12345-0@lo:13:osp-pre-1-1.0 Request processed in 29us (438us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.491227:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 379 00000100:00080000:2.0:1789760751.491246:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760751.491249:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760751.491278:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04886363100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534073600:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760751.491311:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7efa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534073216:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760751.491341:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7efaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534073472:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760751.491353:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7ef8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534073344:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760751.491988:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b645500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3474:1876699534074112:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492040:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.492067:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0488421b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534074624:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492088:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.492116:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:2.0:1789760751.492122:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3474:1876699534073728:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492149:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.492169:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049891fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534074240:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492191:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.492210:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04884219500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534074752:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492229:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.492253:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b644000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534073984:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492275:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.492316:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04884218a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3473:1876699534074496:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492336:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.492360:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1789760751.492366:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3473:1876699534073856:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492388:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.492405:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049891fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3473:1876699534074368:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492423:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760751.492448:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534074112 00000100:00100000:2.0:1789760751.492454:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534074112 00000100:00100000:2.0:1789760751.492482:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534074624 00000100:00100000:2.0:1789760751.492485:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534074624 00000100:00100000:2.0:1789760751.492505:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534074752 00000100:00100000:2.0:1789760751.492508:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534074752 00000100:00100000:2.0:1789760751.492518:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534074496 00000100:00100000:2.0:1789760751.492521:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534074496 00000100:00100000:2.0:1789760751.492532:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534073856 00000100:00100000:2.0:1789760751.492534:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534073856 00000100:00100000:2.0:1789760751.492543:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534074368 00000100:00100000:2.0:1789760751.492545:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534074368 00000100:00100000:2.0:1789760751.492558:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 913 00000100:00100000:2.0:1789760751.492565:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3474:x1876699534074112:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492608:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3474:x1876699534074112:12345-0@lo:400:kworker.0 Request processed in 43us (568us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.492615:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 913 00000100:00100000:2.0:1789760751.492629:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 914 00000100:00100000:2.0:1789760751.492634:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534074624:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492660:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534074624:12345-0@lo:400:kworker.0 Request processed in 26us (571us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.492665:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 914 00000100:00100000:2.0:1789760751.492675:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 915 00000100:00100000:2.0:1789760751.492680:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534074752:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492705:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534074752:12345-0@lo:400:kworker.0 Request processed in 25us (475us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.492711:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 915 00000100:00100000:2.0:1789760751.492722:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 916 00000100:00100000:2.0:1789760751.492727:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7efaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3473:x1876699534074496:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492757:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7efaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3473:x1876699534074496:12345-0@lo:400:kworker.0 Request processed in 29us (420us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.492762:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 916 00000100:00100000:2.0:1789760751.492772:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 917 00000100:00100000:2.0:1789760751.492778:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7efa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3473:x1876699534073856:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492802:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7efa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3473:x1876699534073856:12345-0@lo:400:kworker.0 Request processed in 24us (413us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.492807:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 917 00000100:00100000:2.0:1789760751.492817:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 918 00000100:00100000:2.0:1789760751.492822:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3473:x1876699534074368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492846:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886363100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3473:x1876699534074368:12345-0@lo:400:kworker.0 Request processed in 24us (422us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.492851:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 918 00000100:00100000:2.0:1789760751.492869:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534073728 00000100:00100000:2.0:1789760751.492873:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534073728 00000100:00100000:2.0:1789760751.492899:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 476 00000100:00100000:2.0:1789760751.492905:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cad850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3474:x1876699534073728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.492935:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cad850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3474:x1876699534073728:12345-0@lo:400:kworker.0 Request processed in 30us (785us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.492940:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 476 00000100:00100000:2.0:1789760751.492965:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534074240 00000100:00100000:2.0:1789760751.492968:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534074240 00000100:00100000:2.0:1789760751.492988:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534073984 00000100:00100000:2.0:1789760751.492990:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534073984 00000100:00100000:2.0:1789760751.493015:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 438 00000100:00100000:2.0:1789760751.493021:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534074240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.493051:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534074240:12345-0@lo:400:kworker.0 Request processed in 30us (859us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.493057:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 438 00000100:00100000:2.0:1789760751.493067:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:2.0:1789760751.493072:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3473:x1876699534073984:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.493098:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3473:x1876699534073984:12345-0@lo:400:kworker.0 Request processed in 26us (822us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760751.493104:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00080000:2.0:1789760751.493140:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760751.493144:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760751.493175:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3473:1876699534073856:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.493185:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049891fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3473:1876699534074368:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.493193:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b644000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534073984:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.493201:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04884218a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3473:1876699534074496:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.493229:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3474:1876699534073728:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.493238:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049891fd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534074240:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.493247:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04884219500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534074752:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.493255:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b645500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3474:1876699534074112:0@lo:400:kworker.0 00000100:00100000:2.0:1789760751.493263:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0488421b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534074624:0@lo:400:kworker.0 00000001:00080000:0.0:1789760751.506683:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972668 is committed 00002000:00010000:3.0:1789760751.507592:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21905408 -> 21909504 00002000:00010000:3.0:1789760751.507597:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760751 -> 1789760752 00002000:00010000:3.0:1789760751.507600:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760751 -> 1789760752 00010000:00010000:3.0:1789760751.507615:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5594b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.507634:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5594b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.507643:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5594b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.507650:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760751.507655:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5594b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.507664:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5594b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.507689:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5594b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.507697:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5594b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21905408->21909503] (req 21905408->21909503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.507889:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508367104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42068us (42240us total) trans 4294972668 rc 0/0 00000100:00100000:3.0:1789760751.507899:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5380 00000100:00100000:3.0:1789760751.507911:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730508669026 time 730570513874 state 0x2 ref 0 00000100:00100000:0.0:1789760751.510909:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760751.511071:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508367232 00000100:00100000:3.0:1789760751.511083:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508367232 00000100:00100000:3.0:1789760751.511147:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730508669026 time 730570513874 state 0x0 ref 1 00000100:00100000:3.0:1789760751.511163:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730570774290 time 730570513874 state 0x1 ref 1 00000100:00100000:3.0:1789760751.511229:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730570774290 time 730615938305 state 0x0 ref 1 00000100:00100000:3.0:1789760751.511241:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508367232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760751.511309:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21909504, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:3.0:1789760751.511320:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55952 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.511337:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.511341:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.511357:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55952 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.511374:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55952 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.539574:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21909504 -> 21913600 00010000:00010000:3.0:1789760751.539594:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55952 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.539606:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55952 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.539620:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55952 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.539626:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760751.539631:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55952 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.539640:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55952 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.539662:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55952 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.539670:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55952 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21909504->21913599] (req 21909504->21913599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.539761:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508367232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28517us (28858us total) trans 4294972669 rc 0/0 00000100:00100000:3.0:1789760751.539771:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5381 00000100:00100000:3.0:1789760751.539782:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730570774290 time 730615938305 state 0x2 ref 0 00000001:00080000:0.0:1789760751.539873:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972669 is committed 00000100:00100000:3.0:1789760751.543643:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760751.543727:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508367360 00000100:00100000:0.0:1789760751.543738:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508367360 00000100:00100000:0.0:1789760751.543795:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730570774290 time 730615938305 state 0x0 ref 1 00000100:00100000:0.0:1789760751.543810:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730616198721 time 730615938305 state 0x1 ref 1 00000100:00100000:0.0:1789760751.543840:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730616198721 time 730648551203 state 0x0 ref 1 00000100:00100000:0.0:1789760751.543852:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508367360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760751.543903:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21913600, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:0.0:1789760751.544033:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55959 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.544050:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760751.544055:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760751.544062:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55959 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.544077:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55959 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760751.587192:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21913600 -> 21917696 00010000:00010000:1.0:1789760751.587213:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55959 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.587227:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55959 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.587236:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55959 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.587244:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760751.587249:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55959 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.587258:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55959 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.587281:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55959 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.587308:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55959 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21913600->21917695] (req 21913600->21917695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760751.587414:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508367360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43557us (43775us total) trans 4294972670 rc 0/0 00000100:00100000:1.0:1789760751.587424:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5382 00000100:00100000:1.0:1789760751.587434:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730616198721 time 730648551203 state 0x2 ref 0 00000001:00080000:3.0:1789760751.588805:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972670 is committed 00000100:00100000:2.0:1789760751.590557:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760751.591070:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508367488 00000100:00100000:3.0:1789760751.591080:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508367488 00000100:00100000:3.0:1789760751.591136:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730616198721 time 730648551203 state 0x0 ref 1 00000100:00100000:3.0:1789760751.591151:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730648811619 time 730648551203 state 0x1 ref 1 00000100:00100000:3.0:1789760751.591186:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730648811619 time 730695896783 state 0x0 ref 1 00000100:00100000:3.0:1789760751.591199:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508367488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760751.591248:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21917696, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:3.0:1789760751.591257:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55960 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.591274:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.591278:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.591316:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55960 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.591333:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55960 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760751.617745:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21917696 -> 21921792 00010000:00010000:2.0:1789760751.617766:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55960 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.617779:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55960 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.617794:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55960 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.617802:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760751.617807:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55960 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.617817:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55960 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.617841:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55960 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.617849:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55960 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21917696->21921791] (req 21917696->21921791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.617992:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972671 is committed 00000100:00100000:2.0:1789760751.618055:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508367488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26852us (27500us total) trans 4294972671 rc 0/0 00000100:00100000:2.0:1789760751.618065:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5383 00000100:00100000:2.0:1789760751.618075:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730648811619 time 730695896783 state 0x2 ref 0 00000100:00100000:0.0:1789760751.622922:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760751.623020:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508367616 00000100:00100000:3.0:1789760751.623031:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508367616 00000100:00100000:3.0:1789760751.623088:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730648811619 time 730695896783 state 0x0 ref 1 00000100:00100000:3.0:1789760751.623104:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730696157199 time 730695896783 state 0x1 ref 1 00000100:00100000:3.0:1789760751.623136:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730696157199 time 730727846639 state 0x0 ref 1 00000100:00100000:3.0:1789760751.623151:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508367616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760751.623207:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21921792, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:3.0:1789760751.623217:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55967 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.623234:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.623238:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.623245:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55967 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.623260:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55967 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.657209:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21921792 -> 21925888 00010000:00010000:3.0:1789760751.657241:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55967 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.657256:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55967 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.657265:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55967 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.657273:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760751.657278:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55967 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.657320:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55967 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.657340:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55967 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.657347:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55967 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21921792->21925887] (req 21921792->21925887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.657490:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508367616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34334us (34572us total) trans 4294972672 rc 0/0 00000100:00100000:3.0:1789760751.657500:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5384 00000100:00100000:3.0:1789760751.657511:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730696157199 time 730727846639 state 0x2 ref 0 00000001:00080000:1.0:1789760751.658161:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972672 is committed 00000100:00100000:2.0:1789760751.667806:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760751.668794:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508367744 00000100:00100000:1.0:1789760751.668860:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508367744 00000100:00100000:1.0:1789760751.668920:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730696157199 time 730727846639 state 0x0 ref 1 00000100:00100000:1.0:1789760751.668936:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730728107055 time 730727846639 state 0x1 ref 1 00000100:00100000:1.0:1789760751.669018:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730728107055 time 730773727255 state 0x0 ref 1 00000100:00100000:1.0:1789760751.669031:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508367744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760751.669077:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21925888, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:1.0:1789760751.669086:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5596e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.669100:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760751.669104:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.669110:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5596e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.669124:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5596e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760751.700419:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21925888 -> 21929984 00010000:00010000:2.0:1789760751.700439:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5596e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.700457:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5596e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.700466:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5596e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.700472:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760751.700477:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5596e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.700486:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5596e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.700506:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5596e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.700513:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5596e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21925888->21929983] (req 21925888->21929983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.700606:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508367744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31572us (32805us total) trans 4294972673 rc 0/0 00000100:00100000:2.0:1789760751.700617:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5385 00000100:00100000:2.0:1789760751.700636:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730728107055 time 730773727255 state 0x2 ref 0 00000001:00080000:3.0:1789760751.700990:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972673 is committed 00000100:00100000:2.0:1789760751.707116:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760751.708645:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508367872 00000100:00100000:3.0:1789760751.708658:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508367872 00000100:00100000:2.0:1789760751.708900:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730728107055 time 730773727255 state 0x0 ref 1 00000100:00100000:2.0:1789760751.708970:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730773987671 time 730773727255 state 0x1 ref 1 00000100:00100000:2.0:1789760751.709015:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730773987671 time 730813725380 state 0x0 ref 1 00000100:00100000:2.0:1789760751.709028:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d40050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508367872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760751.709078:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21929984, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:2.0:1789760751.709087:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55975 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.709103:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760751.709107:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760751.709114:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55975 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.709128:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55975 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.744111:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21929984 -> 21934080 00010000:00010000:3.0:1789760751.744129:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55975 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.744143:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55975 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.744153:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55975 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.744165:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760751.744170:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55975 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.744179:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55975 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.744197:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55975 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.744204:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55975 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21929984->21934079] (req 21929984->21934079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.744316:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d40050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508367872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35257us (37175us total) trans 4294972674 rc 0/0 00000100:00100000:3.0:1789760751.744326:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5386 00000100:00100000:3.0:1789760751.744334:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730773987671 time 730813725380 state 0x2 ref 0 00000001:00080000:1.0:1789760751.745275:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972674 is committed 00000100:00100000:1.0:1789760751.756879:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760751.756937:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508368000 00000100:00100000:3.0:1789760751.756948:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508368000 00000100:00100000:3.0:1789760751.757013:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730773987671 time 730813725380 state 0x0 ref 1 00000100:00100000:3.0:1789760751.757028:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730813985796 time 730813725380 state 0x1 ref 1 00000100:00100000:3.0:1789760751.757059:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730813985796 time 730861769587 state 0x0 ref 1 00000100:00100000:3.0:1789760751.757075:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508368000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760751.757121:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21934080, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:3.0:1789760751.757129:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5597c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.757145:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.757149:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.757156:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5597c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.757170:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5597c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.794461:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972675 is committed 00002000:00010000:3.0:1789760751.794613:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21934080 -> 21938176 00010000:00010000:3.0:1789760751.794627:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5597c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.794640:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5597c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.794649:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5597c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.794657:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760751.794661:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5597c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.794671:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5597c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.794689:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5597c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.794697:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5597c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21934080->21938175] (req 21934080->21938175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.794796:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508368000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37718us (37921us total) trans 4294972675 rc 0/0 00000100:00100000:3.0:1789760751.794806:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5387 00000100:00100000:3.0:1789760751.794816:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730813985796 time 730861769587 state 0x2 ref 0 00000100:00100000:2.0:1789760751.804424:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760751.804620:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508368128 00000100:00100000:1.0:1789760751.804630:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508368128 00000100:00100000:1.0:1789760751.804687:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730813985796 time 730861769587 state 0x0 ref 1 00000100:00100000:1.0:1789760751.804701:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730862030003 time 730861769587 state 0x1 ref 1 00000100:00100000:1.0:1789760751.804728:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730862030003 time 730909439271 state 0x0 ref 1 00000100:00100000:1.0:1789760751.804741:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508368128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760751.804791:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21938176, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:1.0:1789760751.804800:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55983 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.804816:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760751.804820:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.804826:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55983 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.804839:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55983 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.849231:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972676 is committed 00002000:00010000:2.0:1789760751.849555:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21938176 -> 21942272 00010000:00010000:2.0:1789760751.849569:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55983 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.849582:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55983 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.849596:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55983 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.849603:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760751.849607:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55983 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.849617:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55983 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.850008:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55983 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.850019:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55983 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21938176->21942271] (req 21938176->21942271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.850217:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508368128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45472us (45797us total) trans 4294972676 rc 0/0 00000100:00100000:2.0:1789760751.850227:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5388 00000100:00100000:2.0:1789760751.850236:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730862030003 time 730909439271 state 0x2 ref 0 00000100:00100000:2.0:1789760751.855597:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760751.855988:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508368256 00000100:00100000:3.0:1789760751.856007:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508368256 00000100:00100000:3.0:1789760751.856058:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730862030003 time 730909439271 state 0x0 ref 1 00000100:00100000:3.0:1789760751.856388:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730909699687 time 730909439271 state 0x1 ref 1 00000100:00100000:3.0:1789760751.856460:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730909699687 time 730961170453 state 0x0 ref 1 00000100:00100000:3.0:1789760751.856473:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508368256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760751.856516:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21942272, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:3.0:1789760751.856524:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5598a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.856538:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760751.856542:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.856548:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5598a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.856560:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5598a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760751.911096:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972677 is committed 00002000:00010000:3.0:1789760751.911196:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21942272 -> 21946368 00010000:00010000:3.0:1789760751.911211:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5598a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.911224:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5598a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.911233:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5598a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.911241:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760751.911246:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5598a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.911257:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5598a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.911278:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5598a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.911319:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5598a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21942272->21946367] (req 21942272->21946367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.911633:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508368256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55156us (56041us total) trans 4294972677 rc 0/0 00000100:00100000:2.0:1789760751.911644:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5389 00000100:00100000:2.0:1789760751.911654:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730909699687 time 730961170453 state 0x2 ref 0 00000100:00100000:1.0:1789760751.918064:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760751.918130:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508368384 00000100:00100000:3.0:1789760751.918140:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508368384 00000100:00100000:3.0:1789760751.918197:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730909699687 time 730961170453 state 0x0 ref 1 00000100:00100000:3.0:1789760751.918212:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730961430869 time 730961170453 state 0x1 ref 1 00000100:00100000:3.0:1789760751.918244:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730961430869 time 731022955066 state 0x0 ref 1 00000100:00100000:3.0:1789760751.918256:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e67800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508368384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760751.918330:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21946368, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:3.0:1789760751.918340:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55991 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.918368:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760751.918373:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.918380:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55991 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.918395:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55991 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760751.966199:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972678 is committed 00002000:00010000:2.0:1789760751.966212:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21946368 -> 21950464 00010000:00010000:2.0:1789760751.966226:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55991 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.966240:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55991 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.966250:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55991 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.966257:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760751.966267:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55991 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.966276:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55991 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.966327:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55991 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760751.966335:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55991 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21946368->21950463] (req 21946368->21950463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760751.966427:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e67800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508368384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48167us (48367us total) trans 4294972678 rc 0/0 00000100:00100000:2.0:1789760751.966442:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5390 00000100:00100000:2.0:1789760751.966451:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730961430869 time 731022955066 state 0x2 ref 0 00000100:00100000:2.0:1789760751.971918:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760751.972036:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508368512 00000100:00100000:1.0:1789760751.972047:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508368512 00000100:00100000:1.0:1789760751.972098:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 730961430869 time 731022955066 state 0x0 ref 1 00000100:00100000:1.0:1789760751.972113:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731023215482 time 731022955066 state 0x1 ref 1 00000100:00100000:1.0:1789760751.972205:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731023215482 time 731076913921 state 0x0 ref 1 00000100:00100000:1.0:1789760751.972219:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508368512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760751.972266:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21950464, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:1.0:1789760751.972276:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55998 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.972320:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760751.972325:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760751.972331:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55998 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760751.972345:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55998 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760751.999199:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21950464 -> 21954560 00010000:00010000:0.0:1789760751.999221:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55998 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.999236:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55998 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.999245:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55998 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.999253:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760751.999258:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55998 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.999267:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55998 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.999300:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55998 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760751.999309:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55998 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21950464->21954559] (req 21950464->21954559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760751.999413:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508368512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27192us (27500us total) trans 4294972679 rc 0/0 00000100:00100000:0.0:1789760751.999532:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5391 00000100:00100000:0.0:1789760751.999544:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731023215482 time 731076913921 state 0x2 ref 0 00000001:00080000:1.0:1789760752.000055:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972679 is committed 00000100:00100000:1.0:1789760752.004170:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760752.004382:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508368640 00000100:00100000:2.0:1789760752.004392:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508368640 00000100:00100000:2.0:1789760752.004507:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731023215482 time 731076913921 state 0x0 ref 1 00000100:00100000:2.0:1789760752.004523:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731077174337 time 731076913921 state 0x1 ref 1 00000100:00100000:2.0:1789760752.004587:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731077174337 time 731109297012 state 0x0 ref 1 00000100:00100000:2.0:1789760752.004601:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508368640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760752.004647:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21954560, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:2.0:1789760752.004656:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5599f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.004671:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760752.004675:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.004681:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5599f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.004696:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5599f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760752.033462:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21954560 -> 21958656 00010000:00010000:0.0:1789760752.033484:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5599f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.033496:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5599f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.033505:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5599f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.033512:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760752.033517:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5599f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.033527:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5599f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.033546:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5599f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.033554:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5599f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21954560->21958655] (req 21954560->21958655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.033644:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508368640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29041us (29478us total) trans 4294972680 rc 0/0 00000100:00100000:0.0:1789760752.033654:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5392 00000100:00100000:0.0:1789760752.033663:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731077174337 time 731109297012 state 0x2 ref 0 00000001:00080000:1.0:1789760752.033939:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972680 is committed 00000100:00100000:2.0:1789760752.037011:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760752.037094:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508368768 00000100:00100000:0.0:1789760752.037104:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508368768 00000100:00100000:0.0:1789760752.037157:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731077174337 time 731109297012 state 0x0 ref 1 00000100:00100000:0.0:1789760752.037173:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731109557428 time 731109297012 state 0x1 ref 1 00000100:00100000:0.0:1789760752.037204:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731109557428 time 731141914556 state 0x0 ref 1 00000100:00100000:0.0:1789760752.037215:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508368768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760752.037259:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21958656, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:0.0:1789760752.037268:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.037309:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760752.037314:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.037320:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.037333:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760752.065758:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21958656 -> 21962752 00000001:00080000:3.0:1789760752.065758:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972681 is committed 00010000:00010000:0.0:1789760752.065780:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.065802:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.065870:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.065877:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760752.065882:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.065891:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.065910:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.065918:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21958656->21962751] (req 21958656->21962751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.066101:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508368768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28883us (29102us total) trans 4294972681 rc 0/0 00000100:00100000:0.0:1789760752.066112:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5393 00000100:00100000:0.0:1789760752.066134:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731109557428 time 731141914556 state 0x2 ref 0 00000100:00100000:2.0:1789760752.069476:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760752.069618:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508368896 00000100:00100000:0.0:1789760752.069627:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508368896 00000100:00100000:0.0:1789760752.069686:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731109557428 time 731141914556 state 0x0 ref 1 00000100:00100000:0.0:1789760752.069700:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731142174972 time 731141914556 state 0x1 ref 1 00000100:00100000:0.0:1789760752.069740:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731142174972 time 731174450958 state 0x0 ref 1 00000100:00100000:0.0:1789760752.069752:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508368896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760752.069815:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21962752, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:0.0:1789760752.069830:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe559ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.069844:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760752.069848:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.069854:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe559ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.069868:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe559ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.107723:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972682 is committed 00002000:00010000:3.0:1789760752.107928:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21962752 -> 21966848 00010000:00010000:3.0:1789760752.107943:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe559ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.107956:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe559ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.107965:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe559ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.107973:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760752.107977:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe559ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.107987:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe559ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.108019:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe559ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.108027:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe559ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21962752->21966847] (req 21962752->21966847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.108118:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508368896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38362us (38647us total) trans 4294972682 rc 0/0 00000100:00100000:3.0:1789760752.108128:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5394 00000100:00100000:3.0:1789760752.108137:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731142174972 time 731174450958 state 0x2 ref 0 00000100:00100000:1.0:1789760752.111546:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760752.111756:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508369024 00000100:00100000:0.0:1789760752.111766:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508369024 00000100:00100000:0.0:1789760752.111819:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731142174972 time 731174450958 state 0x0 ref 1 00000100:00100000:0.0:1789760752.111833:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731174711374 time 731174450958 state 0x1 ref 1 00000100:00100000:0.0:1789760752.112004:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731174711374 time 731216700262 state 0x0 ref 1 00000100:00100000:0.0:1789760752.112018:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508369024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760752.112067:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21966848, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:0.0:1789760752.112077:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.112093:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760752.112097:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.112104:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.112118:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760752.178237:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972683 is committed 00002000:00010000:3.0:1789760752.178354:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21966848 -> 21970944 00010000:00010000:3.0:1789760752.178369:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.178383:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.178392:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.178399:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760752.178405:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.178414:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.178433:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.178440:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe559b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21966848->21970943] (req 21966848->21970943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.178548:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508369024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 66527us (67006us total) trans 4294972683 rc 0/0 00000100:00100000:3.0:1789760752.178559:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5395 00000100:00100000:3.0:1789760752.178569:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731174711374 time 731216700262 state 0x2 ref 0 00000100:00100000:2.0:1789760752.183416:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760752.183576:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508369152 00000100:00100000:0.0:1789760752.183586:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508369152 00000100:00100000:0.0:1789760752.183652:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731174711374 time 731216700262 state 0x0 ref 1 00000100:00100000:0.0:1789760752.183669:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731216960678 time 731216700262 state 0x1 ref 1 00000100:00100000:0.0:1789760752.183698:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731216960678 time 731288408488 state 0x0 ref 1 00000100:00100000:0.0:1789760752.183711:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508369152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760752.183777:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21970944, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:0.0:1789760752.183786:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe559bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.183802:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760752.183807:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.183813:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe559bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.183828:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe559bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760752.226139:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21970944 -> 21975040 00010000:00010000:1.0:1789760752.226161:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe559bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.226172:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe559bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.226182:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe559bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.226188:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760752.226193:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe559bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.226201:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe559bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.226220:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe559bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.226228:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe559bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21970944->21975039] (req 21970944->21975039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.226363:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508369152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42649us (42953us total) trans 4294972684 rc 0/0 00000100:00100000:1.0:1789760752.226375:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5396 00000100:00100000:1.0:1789760752.226385:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731216960678 time 731288408488 state 0x2 ref 0 00000001:00080000:2.0:1789760752.227225:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972684 is committed 00000100:00100000:2.0:1789760752.233213:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760752.233458:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508369280 00000100:00100000:3.0:1789760752.233468:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508369280 00000100:00100000:3.0:1789760752.233526:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731216960678 time 731288408488 state 0x0 ref 1 00000100:00100000:3.0:1789760752.233542:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731288668904 time 731288408488 state 0x1 ref 1 00000100:00100000:3.0:1789760752.233569:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731288668904 time 731338279280 state 0x0 ref 1 00000100:00100000:3.0:1789760752.233581:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508369280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760752.233628:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21975040, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:3.0:1789760752.233637:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe559c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.233660:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760752.233664:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.233670:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe559c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.233684:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe559c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.271147:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972685 is committed 00002000:00010000:2.0:1789760752.271320:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21975040 -> 21979136 00010000:00010000:2.0:1789760752.271335:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe559c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.271356:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe559c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.271366:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe559c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.271373:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760752.271378:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe559c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.271388:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe559c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.271408:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe559c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.271416:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe559c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21975040->21979135] (req 21975040->21979135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.271513:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508369280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37928us (38304us total) trans 4294972685 rc 0/0 00000100:00100000:2.0:1789760752.271523:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5397 00000100:00100000:2.0:1789760752.271532:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731288668904 time 731338279280 state 0x2 ref 0 00000100:00100000:0.0:1789760752.274960:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760752.275082:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508369408 00000100:00100000:3.0:1789760752.275092:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508369408 00000100:00100000:3.0:1789760752.275145:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731288668904 time 731338279280 state 0x0 ref 1 00000100:00100000:3.0:1789760752.275165:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731338539696 time 731338279280 state 0x1 ref 1 00000100:00100000:3.0:1789760752.275195:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731338539696 time 731379905483 state 0x0 ref 1 00000100:00100000:3.0:1789760752.275207:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508369408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760752.275254:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21979136, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:3.0:1789760752.275263:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe559c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.275278:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760752.275282:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.275326:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe559c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.275340:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe559c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.313110:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972686 is committed 00002000:00010000:2.0:1789760752.313207:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21979136 -> 21983232 00010000:00010000:2.0:1789760752.313220:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe559c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.313232:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe559c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.313241:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe559c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.313247:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760752.313252:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe559c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.313261:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe559c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.313281:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe559c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.313423:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe559c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21979136->21983231] (req 21979136->21983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.313616:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508369408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38405us (38658us total) trans 4294972686 rc 0/0 00000100:00100000:3.0:1789760752.313627:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5398 00000100:00100000:3.0:1789760752.313636:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731338539696 time 731379905483 state 0x2 ref 0 00000100:00100000:3.0:1789760752.317282:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760752.317882:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508369536 00000100:00100000:1.0:1789760752.317893:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508369536 00000100:00100000:1.0:1789760752.318072:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731338539696 time 731379905483 state 0x0 ref 1 00000100:00100000:1.0:1789760752.318088:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731380165899 time 731379905483 state 0x1 ref 1 00000100:00100000:1.0:1789760752.318114:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731380165899 time 731422824808 state 0x0 ref 1 00000100:00100000:1.0:1789760752.318128:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508369536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760752.318176:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21983232, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:1.0:1789760752.318185:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.318200:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760752.318204:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.318210:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.318222:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760752.353243:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21983232 -> 21987328 00010000:00010000:1.0:1789760752.353267:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.353280:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.353308:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.353316:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760752.353321:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.353331:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.353360:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.353370:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21983232->21987327] (req 21983232->21987327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.353465:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508369536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35334us (36188us total) trans 4294972687 rc 0/0 00000100:00100000:1.0:1789760752.353478:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5399 00000100:00100000:1.0:1789760752.353488:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731380165899 time 731422824808 state 0x2 ref 0 00000001:00080000:0.0:1789760752.357341:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972687 is committed 00000100:00100000:3.0:1789760752.357786:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760752.358510:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508369664 00000100:00100000:2.0:1789760752.358523:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508369664 00000100:00100000:2.0:1789760752.358579:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731380165899 time 731422824808 state 0x0 ref 1 00000100:00100000:2.0:1789760752.358593:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731423085224 time 731422824808 state 0x1 ref 1 00000100:00100000:2.0:1789760752.358713:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731423085224 time 731463423530 state 0x0 ref 1 00000100:00100000:2.0:1789760752.358726:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508369664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760752.358783:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21987328, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:2.0:1789760752.358792:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.358806:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760752.358810:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.358816:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.358829:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.383580:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760752.391897:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972688 is committed 00002000:00010000:2.0:1789760752.392133:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21987328 -> 21991424 00010000:00010000:2.0:1789760752.392148:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.392160:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.392170:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.392178:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760752.392183:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.392192:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.392210:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.392218:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21987328->21991423] (req 21987328->21991423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.392761:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508369664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34031us (34974us total) trans 4294972688 rc 0/0 00000100:00100000:2.0:1789760752.392772:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5400 00000100:00100000:2.0:1789760752.392780:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731423085224 time 731463423530 state 0x2 ref 0 00000100:00100000:2.0:1789760752.397202:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760752.397324:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508369792 00000100:00100000:3.0:1789760752.397337:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508369792 00000100:00100000:3.0:1789760752.397401:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731423085224 time 731463423530 state 0x0 ref 1 00000100:00100000:3.0:1789760752.397417:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731463683946 time 731463423530 state 0x1 ref 1 00000100:00100000:3.0:1789760752.397444:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731463683946 time 731502154898 state 0x0 ref 1 00000100:00100000:3.0:1789760752.397457:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508369792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760752.397501:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21991424, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:3.0:1789760752.397511:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.397526:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760752.397531:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.397537:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.397551:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.431964:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21991424 -> 21995520 00010000:00010000:3.0:1789760752.431986:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.432007:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.432022:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.432038:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760752.432043:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.432052:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.432074:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.432082:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21991424->21995519] (req 21991424->21995519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.432177:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508369792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34718us (34979us total) trans 4294972689 rc 0/0 00000100:00100000:3.0:1789760752.432187:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5401 00000100:00100000:3.0:1789760752.432197:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731463683946 time 731502154898 state 0x2 ref 0 00000001:00080000:2.0:1789760752.432714:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972689 is committed 00000100:00100000:0.0:1789760752.435573:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760752.435658:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508369920 00000100:00100000:1.0:1789760752.435668:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508369920 00000100:00100000:1.0:1789760752.435726:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731463683946 time 731502154898 state 0x0 ref 1 00000100:00100000:1.0:1789760752.435742:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731502415314 time 731502154898 state 0x1 ref 1 00000100:00100000:1.0:1789760752.435770:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731502415314 time 731540480514 state 0x0 ref 1 00000100:00100000:1.0:1789760752.435782:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508369920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760752.435828:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21995520, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:1.0:1789760752.435838:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe559e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.435854:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760752.435859:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.435865:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe559e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.435880:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe559e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.478443:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972690 is committed 00002000:00010000:1.0:1789760752.478515:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21995520 -> 21999616 00010000:00010000:1.0:1789760752.478529:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe559e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.478542:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe559e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.478551:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe559e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.478559:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760752.478564:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe559e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.478573:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe559e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.478592:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe559e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.478600:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe559e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21995520->21999615] (req 21995520->21999615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.478707:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508369920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42915us (43130us total) trans 4294972690 rc 0/0 00000100:00100000:1.0:1789760752.478718:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5402 00000100:00100000:1.0:1789760752.478727:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731502415314 time 731540480514 state 0x2 ref 0 00000100:00100000:1.0:1789760752.485616:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760752.485813:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508370048 00000100:00100000:2.0:1789760752.485824:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508370048 00000100:00100000:2.0:1789760752.485878:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731502415314 time 731540480514 state 0x0 ref 1 00000100:00100000:2.0:1789760752.485893:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731540740930 time 731540480514 state 0x1 ref 1 00000100:00100000:2.0:1789760752.485923:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731540740930 time 731590634433 state 0x0 ref 1 00000100:00100000:2.0:1789760752.485934:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508370048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760752.485977:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 21999616, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0xa807 00010000:00010000:2.0:1789760752.485986:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.486013:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760752.486018:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.486024:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.486038:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.511534:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887e67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534074880:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789760752.511644:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760752.512213:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534074880 00000100:00100000:1.0:1789760752.512222:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534074880 00000100:00100000:1.0:1789760752.512276:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 440 00000100:00100000:1.0:1789760752.512317:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534074880:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760752.512354:0:7358:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760752.512403:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534074880:12345-0@lo:41:osp-pre-0-1.0 Request processed in 118us (758us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760752.512410:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 440 00000100:00100000:1.0:1789760752.512464:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887e67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534074880:0@lo:41:osp-pre-0-1.0 00000001:00080000:0.0:1789760752.547191:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972691 is committed 00002000:00010000:2.0:1789760752.547393:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 21999616 -> 22003712 00002000:00010000:2.0:1789760752.547398:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760752 -> 1789760753 00002000:00010000:2.0:1789760752.547401:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760752 -> 1789760753 00010000:00010000:2.0:1789760752.547415:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.547429:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.547439:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.547448:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760752.547453:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.547462:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.547485:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.547493:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe559ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [21999616->22003711] (req 21999616->22003711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.547584:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508370048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61647us (61973us total) trans 4294972691 rc 0/0 00000100:00100000:2.0:1789760752.547594:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5403 00000100:00100000:2.0:1789760752.547604:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731540740930 time 731590634433 state 0x2 ref 0 00000100:00100000:0.0:1789760752.551340:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760752.551489:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508370176 00000100:00100000:3.0:1789760752.551499:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508370176 00000100:00100000:3.0:1789760752.551552:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731540740930 time 731590634433 state 0x0 ref 1 00000100:00100000:3.0:1789760752.551566:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731590894849 time 731590634433 state 0x1 ref 1 00000100:00100000:3.0:1789760752.553718:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731590894849 time 731658422856 state 0x0 ref 1 00000100:00100000:3.0:1789760752.553745:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508370176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760752.553801:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22003712, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa807 00010000:00010000:3.0:1789760752.553810:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.553837:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760752.553841:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.553848:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.553863:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.601789:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972692 is committed 00002000:00010000:0.0:1789760752.601934:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22003712 -> 22007808 00010000:00010000:0.0:1789760752.601950:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.601963:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.601972:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.601980:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760752.601985:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.602005:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.602025:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.602033:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe559f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22003712->22007807] (req 22003712->22007807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.602139:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508370176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48398us (50801us total) trans 4294972692 rc 0/0 00000100:00100000:0.0:1789760752.602149:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5404 00000100:00100000:0.0:1789760752.602158:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731590894849 time 731658422856 state 0x2 ref 0 00000100:00100000:0.0:1789760752.612703:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760752.612771:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508370304 00000100:00100000:1.0:1789760752.612782:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508370304 00000100:00100000:1.0:1789760752.612850:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731590894849 time 731658422856 state 0x0 ref 1 00000100:00100000:1.0:1789760752.612866:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731658683272 time 731658422856 state 0x1 ref 1 00000100:00100000:1.0:1789760752.612897:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731658683272 time 731717607926 state 0x0 ref 1 00000100:00100000:1.0:1789760752.612909:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508370304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760752.612956:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22007808, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa807 00010000:00010000:1.0:1789760752.612965:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.612981:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760752.612985:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.613002:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.613016:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760752.649306:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972693 is committed 00002000:00010000:2.0:1789760752.649497:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22007808 -> 22011904 00010000:00010000:2.0:1789760752.649513:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.649526:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.649535:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.649543:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760752.649547:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.649557:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.649576:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.649583:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe559fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22007808->22011903] (req 22007808->22011903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.649667:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508370304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36755us (36969us total) trans 4294972693 rc 0/0 00000100:00100000:2.0:1789760752.649677:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5405 00000100:00100000:2.0:1789760752.649687:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731658683272 time 731717607926 state 0x2 ref 0 00000100:00100000:1.0:1789760752.653668:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760752.653760:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508370432 00000100:00100000:2.0:1789760752.653771:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508370432 00000100:00100000:2.0:1789760752.653925:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731658683272 time 731717607926 state 0x0 ref 1 00000100:00100000:2.0:1789760752.653943:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731717868342 time 731717607926 state 0x1 ref 1 00000100:00100000:2.0:1789760752.654020:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731717868342 time 731758729908 state 0x0 ref 1 00000100:00100000:2.0:1789760752.654033:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e67b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508370432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760752.654153:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22011904, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa807 00010000:00010000:2.0:1789760752.654162:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55a01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.654178:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760752.654254:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.654262:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55a01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.654276:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55a01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760752.709387:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22011904 -> 22016000 00010000:00010000:0.0:1789760752.709406:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55a01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.709419:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55a01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.709427:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55a01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.709435:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760752.709439:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55a01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.709449:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55a01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.709470:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55a01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.709478:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55a01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22011904->22015999] (req 22011904->22015999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.709575:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e67b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508370432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55539us (55911us total) trans 4294972694 rc 0/0 00000100:00100000:0.0:1789760752.709585:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5406 00000100:00100000:0.0:1789760752.709595:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731717868342 time 731758729908 state 0x2 ref 0 00000001:00080000:0.0:1789760752.709635:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972694 is committed 00000100:00100000:0.0:1789760752.714013:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760752.714112:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508370560 00000100:00100000:1.0:1789760752.714122:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508370560 00000100:00100000:1.0:1789760752.714181:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731717868342 time 731758729908 state 0x0 ref 1 00000100:00100000:1.0:1789760752.714196:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731758990324 time 731758729908 state 0x1 ref 1 00000100:00100000:1.0:1789760752.714227:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731758990324 time 731818937331 state 0x0 ref 1 00000100:00100000:1.0:1789760752.714239:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508370560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760752.714325:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22016000, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa807 00010000:00010000:1.0:1789760752.714335:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.714359:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760752.714363:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.714369:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.714383:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.768877:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972695 is committed 00002000:00010000:1.0:1789760752.768980:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22016000 -> 22020096 00010000:00010000:1.0:1789760752.769005:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.769017:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.769026:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.769034:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760752.769038:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.769048:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.769066:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.769075:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22016000->22020095] (req 22016000->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.769170:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508370560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54928us (55162us total) trans 4294972695 rc 0/0 00000100:00100000:1.0:1789760752.769181:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5407 00000100:00100000:1.0:1789760752.769192:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731758990324 time 731818937331 state 0x2 ref 0 00000100:00100000:3.0:1789760752.778639:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760752.783605:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508370688 00000100:00100000:2.0:1789760752.783620:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508370688 00000100:00100000:2.0:1789760752.783688:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731758990324 time 731818937331 state 0x0 ref 1 00000100:00100000:2.0:1789760752.783705:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731819197747 time 731818937331 state 0x1 ref 1 00000100:00100000:2.0:1789760752.783730:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731819197747 time 731888440550 state 0x0 ref 1 00000100:00100000:2.0:1789760752.783743:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508370688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760752.783791:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22020096, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xa807 00010000:00010000:2.0:1789760752.783800:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.783817:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760752.783822:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.783828:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.783843:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.856525:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972696 is committed 00002000:00010000:2.0:1789760752.856824:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22020096 -> 22024192 00002000:00010000:2.0:1789760752.856829:0:14638:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:2.0:1789760752.856841:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.856853:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.856863:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.856874:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760752.856878:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.856886:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.856917:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.856925:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22020096->22024191] (req 22020096->22024191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.857024:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508370688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 73277us (78388us total) trans 4294972696 rc 0/0 00000100:00100000:2.0:1789760752.857035:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5408 00000100:00100000:2.0:1789760752.857043:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731819197747 time 731888440550 state 0x2 ref 0 00000100:00100000:3.0:1789760752.860797:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760752.861066:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508370816 00000100:00100000:0.0:1789760752.861075:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508370816 00000100:00100000:0.0:1789760752.861129:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731819197747 time 731888440550 state 0x0 ref 1 00000100:00100000:0.0:1789760752.861145:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731888700966 time 731888440550 state 0x1 ref 1 00000100:00100000:0.0:1789760752.861176:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731888700966 time 731965886684 state 0x0 ref 1 00000100:00100000:0.0:1789760752.861189:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508370816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760752.861239:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22024192, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:0.0:1789760752.861248:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55a16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.861264:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760752.861268:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760752.861275:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55a16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.861319:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55a16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760752.896006:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22024192 -> 22028288 00010000:00010000:1.0:1789760752.896026:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55a16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.896040:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55a16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.896049:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55a16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.896056:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760752.896060:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55a16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.896070:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55a16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.896094:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55a16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.896101:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55a16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22024192->22028287] (req 22024192->22028287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760752.896193:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508370816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35002us (35400us total) trans 4294972697 rc 0/0 00000100:00100000:1.0:1789760752.896204:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5409 00000100:00100000:1.0:1789760752.896214:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731888700966 time 731965886684 state 0x2 ref 0 00000001:00080000:3.0:1789760752.896679:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972697 is committed 00000100:00100000:0.0:1789760752.905699:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760752.905788:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508370944 00000100:00100000:1.0:1789760752.905798:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508370944 00000100:00100000:1.0:1789760752.905850:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731888700966 time 731965886684 state 0x0 ref 1 00000100:00100000:1.0:1789760752.905865:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731966147100 time 731965886684 state 0x1 ref 1 00000100:00100000:1.0:1789760752.905892:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731966147100 time 732010603478 state 0x0 ref 1 00000100:00100000:1.0:1789760752.905904:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508370944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760752.905952:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22028288, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:1.0:1789760752.905961:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.905977:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760752.905981:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.905996:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.906012:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760752.949139:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972698 is committed 00002000:00010000:0.0:1789760752.949683:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22028288 -> 22032384 00010000:00010000:0.0:1789760752.949699:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.949712:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.949722:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.949729:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760752.949734:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.949748:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.949768:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760752.949776:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55a1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22028288->22032383] (req 22028288->22032383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760752.949959:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508370944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44051us (44265us total) trans 4294972698 rc 0/0 00000100:00100000:0.0:1789760752.949970:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5410 00000100:00100000:0.0:1789760752.949980:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731966147100 time 732010603478 state 0x2 ref 0 00000100:00100000:0.0:1789760752.955640:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760752.955727:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508371072 00000100:00100000:2.0:1789760752.955738:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508371072 00000100:00100000:2.0:1789760752.955793:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 731966147100 time 732010603478 state 0x0 ref 1 00000100:00100000:2.0:1789760752.955808:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732010863894 time 732010603478 state 0x1 ref 1 00000100:00100000:2.0:1789760752.955839:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732010863894 time 732060549613 state 0x0 ref 1 00000100:00100000:2.0:1789760752.955854:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508371072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760752.955903:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22032384, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:2.0:1789760752.955912:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.955940:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760752.955944:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760752.955950:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.955963:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760752.979617:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22032384 -> 22036480 00010000:00010000:2.0:1789760752.979636:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.979648:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.979656:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.979664:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760752.979669:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.979678:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.979697:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760752.979705:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22032384->22036479] (req 22032384->22036479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760752.979796:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508371072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23938us (24160us total) trans 4294972699 rc 0/0 00000100:00100000:2.0:1789760752.979807:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5411 00000100:00100000:2.0:1789760752.979816:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732010863894 time 732060549613 state 0x2 ref 0 00000001:00080000:1.0:1789760752.980144:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972699 is committed 00000100:00100000:3.0:1789760752.983590:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760752.984414:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508371200 00000100:00100000:1.0:1789760752.984425:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508371200 00000100:00100000:1.0:1789760752.984480:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732010863894 time 732060549613 state 0x0 ref 1 00000100:00100000:1.0:1789760752.984571:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732060810029 time 732060549613 state 0x1 ref 1 00000100:00100000:1.0:1789760752.984598:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732060810029 time 732089308614 state 0x0 ref 1 00000100:00100000:1.0:1789760752.984613:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508371200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760752.984660:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22036480, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:1.0:1789760752.984669:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55a2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.984685:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760752.984690:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760752.984696:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55a2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760752.984709:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55a2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.018623:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972700 is committed 00002000:00010000:2.0:1789760753.018625:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22036480 -> 22040576 00010000:00010000:2.0:1789760753.018641:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55a2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.018655:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55a2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.018664:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55a2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.018672:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760753.018677:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55a2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.018686:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55a2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.018706:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55a2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.018713:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe55a2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22036480->22040575] (req 22036480->22040575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.018809:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508371200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34193us (35222us total) trans 4294972700 rc 0/0 00000100:00100000:2.0:1789760753.018818:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5412 00000100:00100000:2.0:1789760753.018827:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732060810029 time 732089308614 state 0x2 ref 0 00000100:00100000:0.0:1789760753.030594:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760753.030654:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508371328 00000100:00100000:1.0:1789760753.030665:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508371328 00000100:00100000:1.0:1789760753.030880:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732060810029 time 732089308614 state 0x0 ref 1 00000100:00100000:1.0:1789760753.030906:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732089569030 time 732089308614 state 0x1 ref 1 00000100:00100000:1.0:1789760753.030936:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732089569030 time 732135646193 state 0x0 ref 1 00000100:00100000:1.0:1789760753.030948:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508371328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760753.031015:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22040576, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:1.0:1789760753.031364:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55a32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.031382:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760753.031387:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.031393:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55a32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.031408:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55a32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760753.032811:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760753.069792:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972701 is committed 00002000:00010000:0.0:1789760753.071370:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22040576 -> 22044672 00010000:00010000:0.0:1789760753.071388:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55a32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.071401:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55a32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.071411:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55a32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.071423:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760753.071428:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55a32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.071437:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55a32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.071458:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55a32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.071466:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55a32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22040576->22044671] (req 22040576->22044671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.071566:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508371328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40616us (40975us total) trans 4294972701 rc 0/0 00000100:00100000:0.0:1789760753.071576:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5413 00000100:00100000:0.0:1789760753.071587:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732089569030 time 732135646193 state 0x2 ref 0 00000100:00100000:0.0:1789760753.076246:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760753.077482:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508371456 00000100:00100000:2.0:1789760753.077495:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508371456 00000100:00100000:2.0:1789760753.077550:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732089569030 time 732135646193 state 0x0 ref 1 00000100:00100000:2.0:1789760753.077565:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732135906609 time 732135646193 state 0x1 ref 1 00000100:00100000:2.0:1789760753.077594:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732135906609 time 732182305122 state 0x0 ref 1 00000100:00100000:2.0:1789760753.077607:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508371456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760753.077653:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22044672, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:2.0:1789760753.077662:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.077682:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.077686:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.077692:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.077707:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.116709:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972702 is committed 00002000:00010000:1.0:1789760753.116784:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22044672 -> 22048768 00010000:00010000:1.0:1789760753.116798:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.116815:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.116824:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.116831:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760753.116836:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.116845:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.116863:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.116871:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55a39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22044672->22048767] (req 22044672->22048767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.117471:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508371456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39859us (41418us total) trans 4294972702 rc 0/0 00000100:00100000:1.0:1789760753.117483:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5414 00000100:00100000:1.0:1789760753.117493:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732135906609 time 732182305122 state 0x2 ref 0 00000100:00100000:3.0:1789760753.122115:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760753.122365:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508371584 00000100:00100000:2.0:1789760753.122374:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508371584 00000100:00100000:2.0:1789760753.122428:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732135906609 time 732182305122 state 0x0 ref 1 00000100:00100000:2.0:1789760753.122444:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732182565538 time 732182305122 state 0x1 ref 1 00000100:00100000:2.0:1789760753.122473:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732182565538 time 732227183643 state 0x0 ref 1 00000100:00100000:2.0:1789760753.122485:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508371584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760753.122532:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22048768, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:2.0:1789760753.122541:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.122561:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.122565:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.122571:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.122584:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760753.168797:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972703 is committed 00002000:00010000:2.0:1789760753.168886:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22048768 -> 22052864 00010000:00010000:2.0:1789760753.168911:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.168925:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.168933:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.168940:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760753.168945:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.168953:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.168971:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.168979:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22048768->22052863] (req 22048768->22052863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.169114:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508371584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46625us (47005us total) trans 4294972703 rc 0/0 00000100:00100000:2.0:1789760753.169125:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5415 00000100:00100000:2.0:1789760753.169134:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732182565538 time 732227183643 state 0x2 ref 0 00000100:00100000:2.0:1789760753.174506:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760753.174582:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508371712 00000100:00100000:0.0:1789760753.174591:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508371712 00000100:00100000:0.0:1789760753.174644:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732182565538 time 732227183643 state 0x0 ref 1 00000100:00100000:0.0:1789760753.174660:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732227444059 time 732227183643 state 0x1 ref 1 00000100:00100000:0.0:1789760753.174724:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732227444059 time 732279434420 state 0x0 ref 1 00000100:00100000:0.0:1789760753.174736:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508371712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760753.174778:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22052864, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:0.0:1789760753.174787:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55a47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.174802:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760753.174806:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.174812:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55a47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.174825:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55a47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760753.231825:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22052864 -> 22056960 00010000:00010000:2.0:1789760753.231845:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55a47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.231857:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55a47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.231866:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55a47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.231874:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760753.231879:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55a47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.231888:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55a47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.231923:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55a47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.231932:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55a47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22052864->22056959] (req 22052864->22056959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.232037:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508371712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57297us (57535us total) trans 4294972704 rc 0/0 00000100:00100000:2.0:1789760753.232047:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5416 00000100:00100000:2.0:1789760753.232057:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732227444059 time 732279434420 state 0x2 ref 0 00000001:00080000:1.0:1789760753.235824:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972704 is committed 00000100:00100000:3.0:1789760753.236768:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760753.236893:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508371840 00000100:00100000:1.0:1789760753.236909:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508371840 00000100:00100000:1.0:1789760753.236956:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732227444059 time 732279434420 state 0x0 ref 1 00000100:00100000:1.0:1789760753.236970:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732279694836 time 732279434420 state 0x1 ref 1 00000100:00100000:1.0:1789760753.237002:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732279694836 time 732341712564 state 0x0 ref 1 00000100:00100000:1.0:1789760753.237013:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508371840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760753.237055:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22056960, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:1.0:1789760753.237064:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55a4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.237077:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760753.237081:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.237087:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55a4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.237100:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55a4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760753.268955:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22056960 -> 22061056 00010000:00010000:2.0:1789760753.268974:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55a4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.268996:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55a4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.269006:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55a4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.269013:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760753.269018:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55a4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.269034:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55a4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.269054:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55a4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.269062:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55a4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22056960->22061055] (req 22056960->22061055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.269155:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508371840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32139us (32388us total) trans 4294972705 rc 0/0 00000100:00100000:2.0:1789760753.269165:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5417 00000100:00100000:2.0:1789760753.269184:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732279694836 time 732341712564 state 0x2 ref 0 00000001:00080000:3.0:1789760753.272339:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972705 is committed 00000100:00100000:1.0:1789760753.276071:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760753.276197:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508371968 00000100:00100000:2.0:1789760753.276208:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508371968 00000100:00100000:2.0:1789760753.276268:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732279694836 time 732341712564 state 0x0 ref 1 00000100:00100000:2.0:1789760753.276283:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732341972980 time 732341712564 state 0x1 ref 1 00000100:00100000:2.0:1789760753.276342:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732341972980 time 732381052710 state 0x0 ref 1 00000100:00100000:2.0:1789760753.276354:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508371968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760753.276400:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22061056, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:2.0:1789760753.276410:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.276434:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.276439:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.276444:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.276458:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.322075:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972706 is committed 00002000:00010000:0.0:1789760753.322313:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22061056 -> 22065152 00010000:00010000:0.0:1789760753.322329:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.322558:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.322567:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.322573:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760753.322577:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.322649:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.322671:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.322678:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe55a55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22061056->22065151] (req 22061056->22065151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.322808:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508371968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46451us (46742us total) trans 4294972706 rc 0/0 00000100:00100000:0.0:1789760753.322817:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5418 00000100:00100000:0.0:1789760753.322827:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732341972980 time 732381052710 state 0x2 ref 0 00000100:00100000:0.0:1789760753.332973:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760753.334054:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508372096 00000100:00100000:2.0:1789760753.334067:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508372096 00000100:00100000:2.0:1789760753.337656:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732341972980 time 732381052710 state 0x0 ref 1 00000100:00100000:2.0:1789760753.337683:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732381313126 time 732381052710 state 0x1 ref 1 00000100:00100000:2.0:1789760753.337732:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732381313126 time 732442441711 state 0x0 ref 1 00000100:00100000:2.0:1789760753.337755:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508372096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760753.337819:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22065152, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:2.0:1789760753.337829:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.337845:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.337850:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.337857:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.337876:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.385087:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972707 is committed 00002000:00010000:2.0:1789760753.385175:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22065152 -> 22069248 00010000:00010000:2.0:1789760753.385191:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.385204:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.385214:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.385221:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760753.385226:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.385235:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.385256:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.385265:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22065152->22069247] (req 22065152->22069247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.385690:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508372096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47737us (52518us total) trans 4294972707 rc 0/0 00000100:00100000:2.0:1789760753.385702:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5419 00000100:00100000:2.0:1789760753.385713:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732381313126 time 732442441711 state 0x2 ref 0 00000100:00100000:3.0:1789760753.389757:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760753.389844:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508372224 00000100:00100000:2.0:1789760753.389855:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508372224 00000100:00100000:2.0:1789760753.389907:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732381313126 time 732442441711 state 0x0 ref 1 00000100:00100000:2.0:1789760753.389922:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732442702127 time 732442441711 state 0x1 ref 1 00000100:00100000:2.0:1789760753.389951:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732442702127 time 732494661047 state 0x0 ref 1 00000100:00100000:2.0:1789760753.389963:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508372224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760753.390023:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22069248, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:2.0:1789760753.390032:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.390049:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.390054:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.390060:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.390074:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.407631:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760753.427940:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972708 is committed 00002000:00010000:2.0:1789760753.430247:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22069248 -> 22073344 00010000:00010000:2.0:1789760753.430268:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.430281:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.430800:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.430808:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760753.430812:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.430822:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.430843:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.430850:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22069248->22073343] (req 22069248->22073343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760753.431019:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508372224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41051us (41266us total) trans 4294972708 rc 0/0 00000100:00100000:2.0:1789760753.431035:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5420 00000100:00100000:2.0:1789760753.431044:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732442702127 time 732494661047 state 0x2 ref 0 00000100:00100000:1.0:1789760753.435721:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760753.436515:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508372352 00000100:00100000:0.0:1789760753.436595:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508372352 00000100:00100000:0.0:1789760753.436660:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732442702127 time 732494661047 state 0x0 ref 1 00000100:00100000:0.0:1789760753.436675:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732494921463 time 732494661047 state 0x1 ref 1 00000100:00100000:0.0:1789760753.436758:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732494921463 time 732541468254 state 0x0 ref 1 00000100:00100000:0.0:1789760753.436772:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508372352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760753.436821:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22073344, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:0.0:1789760753.436831:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55a6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.436853:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760753.436858:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.436864:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55a6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.436879:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55a6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.473799:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972709 is committed 00002000:00010000:3.0:1789760753.473895:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22073344 -> 22077440 00010000:00010000:3.0:1789760753.473910:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55a6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.473923:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55a6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.473932:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55a6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.473939:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760753.473944:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55a6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.473953:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55a6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.474049:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55a6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.474057:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55a6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22073344->22077439] (req 22073344->22077439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.474148:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508372352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37373us (38430us total) trans 4294972709 rc 0/0 00000100:00100000:3.0:1789760753.474159:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5421 00000100:00100000:3.0:1789760753.474169:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732494921463 time 732541468254 state 0x2 ref 0 00000100:00100000:1.0:1789760753.482586:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760753.484268:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508372480 00000100:00100000:2.0:1789760753.484282:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508372480 00000100:00100000:2.0:1789760753.484479:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732494921463 time 732541468254 state 0x0 ref 1 00000100:00100000:2.0:1789760753.484493:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732541728670 time 732541468254 state 0x1 ref 1 00000100:00100000:2.0:1789760753.484526:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732541728670 time 732589237088 state 0x0 ref 1 00000100:00100000:2.0:1789760753.484542:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e67480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508372480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760753.484590:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22077440, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0xb007 00010000:00010000:2.0:1789760753.484599:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.484614:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.484619:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.484625:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.484642:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.516854:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972710 is committed 00002000:00010000:3.0:1789760753.516944:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22077440 -> 22081536 00002000:00010000:3.0:1789760753.516949:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760753 -> 1789760754 00002000:00010000:3.0:1789760753.516952:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760753 -> 1789760754 00010000:00010000:3.0:1789760753.516965:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.516979:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.517067:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.517075:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760753.517080:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.517160:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.517183:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.517191:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55a71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22077440->22081535] (req 22077440->22081535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.517461:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e67480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508372480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32916us (34879us total) trans 4294972710 rc 0/0 00000100:00100000:3.0:1789760753.517473:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5422 00000100:00100000:3.0:1789760753.517482:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732541728670 time 732589237088 state 0x2 ref 0 00000100:00100000:0.0:1789760753.531872:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760753.532706:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508372608 00000100:00100000:2.0:1789760753.532717:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508372608 00000100:00100000:2.0:1789760753.532771:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732541728670 time 732589237088 state 0x0 ref 1 00000100:00100000:2.0:1789760753.532785:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732589497504 time 732589237088 state 0x1 ref 1 00000100:00100000:2.0:1789760753.532817:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732589497504 time 732637527024 state 0x0 ref 1 00000100:00100000:2.0:1789760753.532830:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508372608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760753.532877:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22081536, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:2.0:1789760753.532886:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.532902:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.532906:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.532913:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.532925:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.599832:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972711 is committed 00002000:00010000:3.0:1789760753.601362:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22081536 -> 22085632 00010000:00010000:3.0:1789760753.601382:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.601394:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.601404:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.601411:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760753.601416:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.601425:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.601446:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.601458:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55a78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22081536->22085631] (req 22081536->22085631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.601564:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508372608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 68730us (69697us total) trans 4294972711 rc 0/0 00000100:00100000:3.0:1789760753.601576:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5423 00000100:00100000:3.0:1789760753.601588:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732589497504 time 732637527024 state 0x2 ref 0 00000100:00100000:1.0:1789760753.606040:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760753.606141:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508372736 00000100:00100000:3.0:1789760753.606151:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508372736 00000100:00100000:3.0:1789760753.606205:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732589497504 time 732637527024 state 0x0 ref 1 00000100:00100000:3.0:1789760753.606220:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732637787440 time 732637527024 state 0x1 ref 1 00000100:00100000:2.0:1789760753.606535:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732637787440 time 732711242973 state 0x0 ref 1 00000100:00100000:2.0:1789760753.606553:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508372736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760753.606606:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22085632, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:2.0:1789760753.606615:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55a7f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.606635:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.606640:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.606646:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55a7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.606661:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55a7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.633341:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22085632 -> 22089728 00010000:00010000:3.0:1789760753.633361:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55a7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.633375:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55a7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.633383:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55a7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.633390:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760753.633395:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55a7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.633406:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55a7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.633425:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55a7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.633433:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55a7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22085632->22089727] (req 22085632->22089727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.633515:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508372736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26962us (27533us total) trans 4294972712 rc 0/0 00000100:00100000:3.0:1789760753.633525:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5424 00000100:00100000:3.0:1789760753.633534:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732637787440 time 732711242973 state 0x2 ref 0 00000001:00080000:1.0:1789760753.634180:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972712 is committed 00000100:00100000:1.0:1789760753.638172:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760753.638275:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508372864 00000100:00100000:3.0:1789760753.638460:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508372864 00000100:00100000:3.0:1789760753.638521:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732637787440 time 732711242973 state 0x0 ref 1 00000100:00100000:3.0:1789760753.638536:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732711503389 time 732711242973 state 0x1 ref 1 00000100:00100000:3.0:1789760753.638593:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732711503389 time 732743302843 state 0x0 ref 1 00000100:00100000:3.0:1789760753.638605:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508372864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760753.638649:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22089728, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:3.0:1789760753.638657:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55a86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.638673:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760753.638677:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.638684:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55a86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.638697:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55a86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.665569:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972713 is committed 00002000:00010000:0.0:1789760753.665823:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22089728 -> 22093824 00010000:00010000:0.0:1789760753.665839:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55a86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.665851:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55a86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.665859:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55a86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.665867:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760753.665872:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55a86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.665886:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55a86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.665906:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55a86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.665915:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55a86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22089728->22093823] (req 22089728->22093823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.666013:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508372864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27404us (27845us total) trans 4294972713 rc 0/0 00000100:00100000:0.0:1789760753.666023:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5425 00000100:00100000:0.0:1789760753.666117:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732711503389 time 732743302843 state 0x2 ref 0 00000100:00100000:0.0:1789760753.671135:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760753.671594:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508372992 00000100:00100000:3.0:1789760753.671605:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508372992 00000100:00100000:3.0:1789760753.671665:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732711503389 time 732743302843 state 0x0 ref 1 00000100:00100000:3.0:1789760753.671678:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732743563259 time 732743302843 state 0x1 ref 1 00000100:00100000:3.0:1789760753.671749:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732743563259 time 732776458789 state 0x0 ref 1 00000100:00100000:3.0:1789760753.671761:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508372992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760753.671804:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22093824, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:3.0:1789760753.671812:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55a8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.671826:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760753.671830:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.671835:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55a8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.671853:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55a8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.699578:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22093824 -> 22097920 00010000:00010000:3.0:1789760753.699599:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55a8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.699613:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55a8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.699622:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55a8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.699628:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760753.699633:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55a8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.699640:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55a8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.699659:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55a8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.699663:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55a8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22093824->22097919] (req 22093824->22097919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.699772:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972714 is committed 00000100:00100000:3.0:1789760753.699825:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508372992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28062us (28697us total) trans 4294972714 rc 0/0 00000100:00100000:3.0:1789760753.699836:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5426 00000100:00100000:3.0:1789760753.699846:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732743563259 time 732776458789 state 0x2 ref 0 00000100:00100000:1.0:1789760753.703869:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760753.704167:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508373120 00000100:00100000:3.0:1789760753.704177:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508373120 00000100:00100000:3.0:1789760753.704234:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732743563259 time 732776458789 state 0x0 ref 1 00000100:00100000:3.0:1789760753.704246:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732776719205 time 732776458789 state 0x1 ref 1 00000100:00100000:3.0:1789760753.704268:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732776719205 time 732808979912 state 0x0 ref 1 00000100:00100000:3.0:1789760753.704276:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508373120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760753.705004:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22097920, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:3.0:1789760753.705015:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55a94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.705031:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760753.705035:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.705041:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55a94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.705055:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55a94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.738147:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972715 is committed 00002000:00010000:0.0:1789760753.738253:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22097920 -> 22102016 00010000:00010000:0.0:1789760753.738267:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55a94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.738279:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55a94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.738310:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55a94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.738321:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760753.738326:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55a94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.738346:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55a94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.738366:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55a94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.738373:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55a94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22097920->22102015] (req 22097920->22102015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.738474:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508373120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34192us (34611us total) trans 4294972715 rc 0/0 00000100:00100000:0.0:1789760753.738484:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5427 00000100:00100000:0.0:1789760753.738493:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732776719205 time 732808979912 state 0x2 ref 0 00000100:00100000:2.0:1789760753.743191:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760753.747489:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508373248 00000100:00100000:3.0:1789760753.747506:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508373248 00000100:00100000:3.0:1789760753.747571:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732776719205 time 732808979912 state 0x0 ref 1 00000100:00100000:3.0:1789760753.747588:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732809240328 time 732808979912 state 0x1 ref 1 00000100:00100000:2.0:1789760753.747733:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732809240328 time 732852440363 state 0x0 ref 1 00000100:00100000:2.0:1789760753.747751:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508373248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760753.747801:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22102016, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:2.0:1789760753.747809:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55a9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.747825:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760753.747830:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760753.747837:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55a9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760753.747851:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55a9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760753.805997:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972716 is committed 00002000:00010000:1.0:1789760753.806220:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22102016 -> 22106112 00010000:00010000:1.0:1789760753.806238:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55a9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.806253:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55a9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.806264:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55a9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.806272:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760753.806277:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55a9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.806314:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55a9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.806346:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55a9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.806354:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55a9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22102016->22106111] (req 22102016->22106111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.806456:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508373248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58705us (63271us total) trans 4294972716 rc 0/0 00000100:00100000:1.0:1789760753.806466:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5428 00000100:00100000:1.0:1789760753.806476:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732809240328 time 732852440363 state 0x2 ref 0 00000100:00100000:3.0:1789760753.816796:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760753.821343:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508373376 00000100:00100000:2.0:1789760753.821360:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508373376 00000100:00100000:3.0:1789760753.821594:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732809240328 time 732852440363 state 0x0 ref 1 00000100:00100000:3.0:1789760753.821613:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732852700779 time 732852440363 state 0x1 ref 1 00000100:00100000:3.0:1789760753.821765:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732852700779 time 732926357987 state 0x0 ref 1 00000100:00100000:3.0:1789760753.821782:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508373376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760753.821831:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22106112, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:3.0:1789760753.821840:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55aa2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.821857:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760753.821861:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.821868:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55aa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.821882:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55aa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760753.905855:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22106112 -> 22110208 00010000:00010000:1.0:1789760753.905875:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55aa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.905888:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55aa2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.905897:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55aa2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.905905:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760753.905910:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55aa2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.905920:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55aa2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.905942:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55aa2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.905955:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55aa2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22106112->22110207] (req 22106112->22110207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.906183:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508373376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 84400us (89392us total) trans 4294972717 rc 0/0 00000100:00100000:1.0:1789760753.906194:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5429 00000100:00100000:1.0:1789760753.906205:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732852700779 time 732926357987 state 0x2 ref 0 00000001:00080000:2.0:1789760753.906852:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972717 is committed 00000100:00100000:3.0:1789760753.912684:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760753.912785:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508373504 00000100:00100000:0.0:1789760753.912796:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508373504 00000100:00100000:0.0:1789760753.912850:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732852700779 time 732926357987 state 0x0 ref 1 00000100:00100000:0.0:1789760753.912866:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732926618403 time 732926357987 state 0x1 ref 1 00000100:00100000:0.0:1789760753.912901:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732926618403 time 733017611320 state 0x0 ref 1 00000100:00100000:0.0:1789760753.912918:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508373504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760753.912969:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22110208, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:0.0:1789760753.912992:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55aa9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.913009:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760753.913014:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760753.913028:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55aa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.913043:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55aa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760753.948685:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972718 is committed 00002000:00010000:0.0:1789760753.948785:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22110208 -> 22114304 00010000:00010000:0.0:1789760753.948800:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55aa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.948811:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55aa9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.948819:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55aa9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.948826:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760753.948830:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55aa9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.948839:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55aa9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.948858:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55aa9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760753.948865:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55aa9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22110208->22114303] (req 22110208->22114303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760753.948959:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508373504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36039us (36279us total) trans 4294972718 rc 0/0 00000100:00100000:0.0:1789760753.948969:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5430 00000100:00100000:0.0:1789760753.949476:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732926618403 time 733017611320 state 0x2 ref 0 00000100:00100000:3.0:1789760753.954162:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760753.954233:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508373632 00000100:00100000:1.0:1789760753.954247:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508373632 00000100:00100000:1.0:1789760753.954348:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 732926618403 time 733017611320 state 0x0 ref 1 00000100:00100000:1.0:1789760753.954364:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733017871736 time 733017611320 state 0x1 ref 1 00000100:00100000:1.0:1789760753.954399:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733017871736 time 733059109497 state 0x0 ref 1 00000100:00100000:1.0:1789760753.954414:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508373632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760753.954473:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22114304, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:1.0:1789760753.954482:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.954497:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760753.954502:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760753.954507:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.954521:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760753.996934:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22114304 -> 22118400 00010000:00010000:1.0:1789760753.996955:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.996969:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.996988:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.996997:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760753.997002:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.997011:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.997030:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760753.997038:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22114304->22118399] (req 22114304->22118399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760753.997155:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508373632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42740us (43003us total) trans 4294972719 rc 0/0 00000100:00100000:1.0:1789760753.997166:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5431 00000100:00100000:1.0:1789760753.997175:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733017871736 time 733059109497 state 0x2 ref 0 00000001:00080000:0.0:1789760753.997709:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972719 is committed 00000100:00100000:0.0:1789760754.005546:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760754.005662:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508373760 00000100:00100000:1.0:1789760754.005672:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508373760 00000100:00100000:1.0:1789760754.005727:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733017871736 time 733059109497 state 0x0 ref 1 00000100:00100000:1.0:1789760754.005742:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733059369913 time 733059109497 state 0x1 ref 1 00000100:00100000:1.0:1789760754.005775:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733059369913 time 733110485431 state 0x0 ref 1 00000100:00100000:1.0:1789760754.005787:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508373760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760754.005834:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22118400, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:1.0:1789760754.005842:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.005858:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760754.005862:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.005869:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.005882:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.037511:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972720 is committed 00002000:00010000:2.0:1789760754.037639:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22118400 -> 22122496 00010000:00010000:2.0:1789760754.037655:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.037669:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.037679:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.037687:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760754.037693:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.037702:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.037722:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.037730:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ab7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22118400->22122495] (req 22118400->22122495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.037863:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508373760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32072us (32321us total) trans 4294972720 rc 0/0 00000100:00100000:2.0:1789760754.037878:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5432 00000100:00100000:2.0:1789760754.037889:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733059369913 time 733110485431 state 0x2 ref 0 00000100:00100000:2.0:1789760754.052483:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760754.052615:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508373888 00000100:00100000:0.0:1789760754.052642:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508373888 00000100:00100000:0.0:1789760754.052706:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733059369913 time 733110485431 state 0x0 ref 1 00000100:00100000:0.0:1789760754.052721:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733110745847 time 733110485431 state 0x1 ref 1 00000100:00100000:0.0:1789760754.052754:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733110745847 time 733157464248 state 0x0 ref 1 00000100:00100000:0.0:1789760754.052769:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508373888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760754.052821:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22122496, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:0.0:1789760754.052830:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55abe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.052844:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760754.052849:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.052855:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55abe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.052869:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55abe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.145679:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972721 is committed 00002000:00010000:2.0:1789760754.146183:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22122496 -> 22126592 00010000:00010000:2.0:1789760754.146200:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55abe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.146211:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55abe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.146220:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55abe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.146226:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760754.146231:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55abe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.146238:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55abe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.146257:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55abe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.146264:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55abe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22122496->22126591] (req 22122496->22126591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.146406:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508373888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 93636us (93934us total) trans 4294972721 rc 0/0 00000100:00100000:2.0:1789760754.146415:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5433 00000100:00100000:2.0:1789760754.146425:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733110745847 time 733157464248 state 0x2 ref 0 00000100:00100000:3.0:1789760754.155204:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760754.155315:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508374016 00000100:00100000:1.0:1789760754.155326:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508374016 00000100:00100000:1.0:1789760754.155377:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733110745847 time 733157464248 state 0x0 ref 1 00000100:00100000:1.0:1789760754.155391:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733157724664 time 733157464248 state 0x1 ref 1 00000100:00100000:1.0:1789760754.155504:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733157724664 time 733260214686 state 0x0 ref 1 00000100:00100000:1.0:1789760754.155517:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508374016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760754.155563:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22126592, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:1.0:1789760754.155571:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ac5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.155584:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760754.155588:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.155594:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ac5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.155605:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ac5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.175276:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972722 is committed 00002000:00010000:1.0:1789760754.175447:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22126592 -> 22130688 00010000:00010000:1.0:1789760754.175462:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ac5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.175480:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ac5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.175488:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ac5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.175495:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760754.175499:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ac5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.175508:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ac5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.175529:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ac5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.175536:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ac5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22126592->22130687] (req 22126592->22130687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.175638:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508374016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20117us (20437us total) trans 4294972722 rc 0/0 00000100:00100000:1.0:1789760754.175647:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5434 00000100:00100000:1.0:1789760754.175657:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733157724664 time 733260214686 state 0x2 ref 0 00000100:00100000:0.0:1789760754.178946:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760754.179320:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508374144 00000100:00100000:2.0:1789760754.179339:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508374144 00000100:00100000:2.0:1789760754.179389:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733157724664 time 733260214686 state 0x0 ref 1 00000100:00100000:2.0:1789760754.179402:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733260475102 time 733260214686 state 0x1 ref 1 00000100:00100000:2.0:1789760754.179427:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733260475102 time 733284138221 state 0x0 ref 1 00000100:00100000:2.0:1789760754.179437:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508374144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760754.179483:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22130688, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:2.0:1789760754.179490:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55acc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.179505:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760754.179510:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.179515:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55acc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.179527:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55acc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.206738:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972723 is committed 00002000:00010000:3.0:1789760754.208486:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22130688 -> 22134784 00010000:00010000:3.0:1789760754.208504:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55acc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.208519:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55acc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.208529:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55acc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.208536:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760754.208541:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55acc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.208551:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55acc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.208570:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55acc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.208648:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55acc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22130688->22134783] (req 22130688->22134783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.208766:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508374144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29325us (29826us total) trans 4294972723 rc 0/0 00000100:00100000:3.0:1789760754.208777:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5435 00000100:00100000:3.0:1789760754.208787:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733260475102 time 733284138221 state 0x2 ref 0 00000100:00100000:3.0:1789760754.211741:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760754.212396:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508374272 00000100:00100000:2.0:1789760754.212408:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508374272 00000100:00100000:2.0:1789760754.212516:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733260475102 time 733284138221 state 0x0 ref 1 00000100:00100000:2.0:1789760754.212532:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733284398637 time 733284138221 state 0x1 ref 1 00000100:00100000:2.0:1789760754.212556:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733284398637 time 733317267024 state 0x0 ref 1 00000100:00100000:2.0:1789760754.212570:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508374272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760754.212618:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22134784, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:2.0:1789760754.212627:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55ad3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.212641:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760754.212646:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.212653:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55ad3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.212666:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55ad3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.240865:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972724 is committed 00002000:00010000:3.0:1789760754.241025:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22134784 -> 22138880 00010000:00010000:3.0:1789760754.241103:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55ad3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.241116:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55ad3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.241125:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55ad3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.241132:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760754.241137:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55ad3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.241146:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55ad3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.241164:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55ad3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.241176:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55ad3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22134784->22138879] (req 22134784->22138879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.241733:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508374272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29158us (29992us total) trans 4294972724 rc 0/0 00000100:00100000:3.0:1789760754.241743:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5436 00000100:00100000:3.0:1789760754.241753:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733284398637 time 733317267024 state 0x2 ref 0 00000100:00100000:0.0:1789760754.246687:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760754.247129:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508374400 00000100:00100000:1.0:1789760754.247140:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508374400 00000100:00100000:1.0:1789760754.247201:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733284398637 time 733317267024 state 0x0 ref 1 00000100:00100000:1.0:1789760754.247216:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733317527440 time 733317267024 state 0x1 ref 1 00000100:00100000:1.0:1789760754.247247:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733317527440 time 733351958306 state 0x0 ref 1 00000100:00100000:1.0:1789760754.247259:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508374400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760754.247594:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22138880, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:1.0:1789760754.247604:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ada lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.247620:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760754.247624:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.247631:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ada lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.247647:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ada lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.276201:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972725 is committed 00002000:00010000:3.0:1789760754.276903:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22138880 -> 22142976 00010000:00010000:3.0:1789760754.276921:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ada lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.276934:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ada lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.276943:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ada lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.276950:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760754.276955:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ada lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.276964:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ada lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.276998:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ada lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.277007:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55ada lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22138880->22142975] (req 22138880->22142975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.277100:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508374400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29838us (30523us total) trans 4294972725 rc 0/0 00000100:00100000:3.0:1789760754.277111:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5437 00000100:00100000:3.0:1789760754.277120:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733317527440 time 733351958306 state 0x2 ref 0 00000100:00100000:0.0:1789760754.283470:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760754.283639:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508374528 00000100:00100000:3.0:1789760754.283650:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508374528 00000100:00100000:3.0:1789760754.283705:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733317527440 time 733351958306 state 0x0 ref 1 00000100:00100000:3.0:1789760754.283721:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733352218722 time 733351958306 state 0x1 ref 1 00000100:00100000:3.0:1789760754.283756:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733352218722 time 733388466526 state 0x0 ref 1 00000100:00100000:3.0:1789760754.283769:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508374528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760754.283826:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22142976, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:3.0:1789760754.283835:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55ae1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.283851:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.283857:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.283863:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55ae1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.283877:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55ae1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760754.308485:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22142976 -> 22147072 00010000:00010000:0.0:1789760754.308504:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55ae1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.308517:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55ae1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760754.308520:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972726 is committed 00010000:00010000:0.0:1789760754.308526:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55ae1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.308533:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760754.308538:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55ae1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.308547:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55ae1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.308692:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55ae1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.308701:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55ae1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22142976->22147071] (req 22142976->22147071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.309012:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508374528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25031us (25339us total) trans 4294972726 rc 0/0 00000100:00100000:0.0:1789760754.309023:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5438 00000100:00100000:0.0:1789760754.309033:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733352218722 time 733388466526 state 0x2 ref 0 00000100:00100000:2.0:1789760754.312479:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760754.312558:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508374656 00000100:00100000:3.0:1789760754.312568:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508374656 00000100:00100000:3.0:1789760754.312628:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733352218722 time 733388466526 state 0x0 ref 1 00000100:00100000:3.0:1789760754.312642:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733388726942 time 733388466526 state 0x1 ref 1 00000100:00100000:3.0:1789760754.312672:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733388726942 time 733417383186 state 0x0 ref 1 00000100:00100000:3.0:1789760754.312681:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508374656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760754.312720:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22147072, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:3.0:1789760754.312727:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55ae8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.312738:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.312742:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.312746:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55ae8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.312755:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55ae8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760754.352055:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22147072 -> 22151168 00010000:00010000:0.0:1789760754.352073:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55ae8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.352087:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55ae8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.352096:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55ae8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.352110:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760754.352115:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55ae8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.352124:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55ae8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.352144:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55ae8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.352153:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55ae8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22147072->22151167] (req 22147072->22151167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.352248:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508374656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39563us (39773us total) trans 4294972727 rc 0/0 00000100:00100000:0.0:1789760754.352258:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5439 00000100:00100000:0.0:1789760754.352269:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733388726942 time 733417383186 state 0x2 ref 0 00000001:00080000:1.0:1789760754.354535:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972727 is committed 00000100:00100000:0.0:1789760754.356505:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760754.356755:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508374784 00000100:00100000:3.0:1789760754.356765:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508374784 00000100:00100000:3.0:1789760754.356821:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733388726942 time 733417383186 state 0x0 ref 1 00000100:00100000:3.0:1789760754.356836:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733417643602 time 733417383186 state 0x1 ref 1 00000100:00100000:3.0:1789760754.356867:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733417643602 time 733461577663 state 0x0 ref 1 00000100:00100000:3.0:1789760754.356879:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508374784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760754.357042:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22151168, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:3.0:1789760754.357051:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55aef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.357066:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.357070:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.357078:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55aef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.357093:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55aef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760754.389098:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22151168 -> 22155264 00010000:00010000:1.0:1789760754.389118:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55aef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.389133:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55aef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.389142:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55aef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.389148:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760754.389152:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55aef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.389160:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55aef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.389178:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55aef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.389187:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55aef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22151168->22155263] (req 22151168->22155263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.389281:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508374784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32398us (32779us total) trans 4294972728 rc 0/0 00000100:00100000:1.0:1789760754.389323:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5440 00000100:00100000:1.0:1789760754.389344:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733417643602 time 733461577663 state 0x2 ref 0 00000001:00080000:0.0:1789760754.389499:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972728 is committed 00000100:00100000:2.0:1789760754.395134:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760754.395228:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508374912 00000100:00100000:0.0:1789760754.395238:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508374912 00000100:00100000:0.0:1789760754.395334:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733417643602 time 733461577663 state 0x0 ref 1 00000100:00100000:0.0:1789760754.395351:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733461838079 time 733461577663 state 0x1 ref 1 00000100:00100000:0.0:1789760754.395378:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733461838079 time 733500088291 state 0x0 ref 1 00000100:00100000:0.0:1789760754.395390:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508374912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760754.395440:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22155264, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:0.0:1789760754.395449:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55af6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.395465:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760754.395470:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.395476:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55af6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.395491:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55af6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760754.421362:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22155264 -> 22159360 00010000:00010000:0.0:1789760754.421383:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55af6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.421397:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55af6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.421407:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55af6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.421414:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760754.421420:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55af6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.421429:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55af6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.421451:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55af6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.421458:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55af6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22155264->22159359] (req 22155264->22159359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.421694:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508374912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26300us (26565us total) trans 4294972729 rc 0/0 00000100:00100000:0.0:1789760754.421705:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5441 00000100:00100000:0.0:1789760754.421715:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733461838079 time 733500088291 state 0x2 ref 0 00000001:00080000:0.0:1789760754.421757:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972729 is committed 00000100:00100000:2.0:1789760754.426031:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760754.426112:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508375040 00000100:00100000:0.0:1789760754.426123:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508375040 00000100:00100000:0.0:1789760754.426178:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733461838079 time 733500088291 state 0x0 ref 1 00000100:00100000:0.0:1789760754.426193:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733500348707 time 733500088291 state 0x1 ref 1 00000100:00100000:0.0:1789760754.426226:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733500348707 time 733530936016 state 0x0 ref 1 00000100:00100000:0.0:1789760754.426239:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508375040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760754.426326:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22159360, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:0.0:1789760754.426345:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55afd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.426362:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760754.426366:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.426372:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55afd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.426386:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55afd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.436227:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760754.463318:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972730 is committed 00002000:00010000:3.0:1789760754.463470:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22159360 -> 22163456 00010000:00010000:3.0:1789760754.463591:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55afd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.463612:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55afd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.463622:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55afd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.463629:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760754.463634:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55afd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.463643:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55afd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.463663:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55afd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.463671:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55afd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22159360->22163455] (req 22159360->22163455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.463762:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508375040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37521us (37737us total) trans 4294972730 rc 0/0 00000100:00100000:3.0:1789760754.463772:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5442 00000100:00100000:3.0:1789760754.463781:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733500348707 time 733530936016 state 0x2 ref 0 00000100:00100000:3.0:1789760754.467838:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760754.468004:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508375168 00000100:00100000:1.0:1789760754.468014:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508375168 00000100:00100000:1.0:1789760754.468068:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733500348707 time 733530936016 state 0x0 ref 1 00000100:00100000:1.0:1789760754.468084:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733531196432 time 733530936016 state 0x1 ref 1 00000100:00100000:1.0:1789760754.468145:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733531196432 time 733572854979 state 0x0 ref 1 00000100:00100000:1.0:1789760754.468157:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d42c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508375168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760754.468205:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22163456, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0xb007 00010000:00010000:1.0:1789760754.468215:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.468231:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760754.468236:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.468242:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.468257:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.501776:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972731 is committed 00002000:00010000:1.0:1789760754.501881:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22163456 -> 22167552 00002000:00010000:1.0:1789760754.501885:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760754 -> 1789760755 00002000:00010000:1.0:1789760754.501888:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760754 -> 1789760755 00010000:00010000:1.0:1789760754.501901:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.501918:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.501927:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.501934:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760754.501938:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.501947:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b04 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.501967:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b04 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.501983:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b04 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22163456->22167551] (req 22163456->22167551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.502107:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d42c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508375168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33947us (34275us total) trans 4294972731 rc 0/0 00000100:00100000:1.0:1789760754.502118:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5443 00000100:00100000:1.0:1789760754.502128:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733531196432 time 733572854979 state 0x2 ref 0 00000100:00100000:2.0:1789760754.509661:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760754.509759:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508375296 00000100:00100000:0.0:1789760754.509769:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508375296 00000100:00100000:0.0:1789760754.509823:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733531196432 time 733572854979 state 0x0 ref 1 00000100:00100000:0.0:1789760754.509839:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733573115395 time 733572854979 state 0x1 ref 1 00000100:00100000:0.0:1789760754.509916:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733573115395 time 733614624926 state 0x0 ref 1 00000100:00100000:0.0:1789760754.509930:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508375296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760754.510002:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22167552, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:0.0:1789760754.510012:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.510028:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760754.510034:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.510040:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.510055:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.558679:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760754.558703:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887e64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534075008:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760754.558802:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760754.558871:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534075008 00000100:00100000:1.0:1789760754.558879:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534075008 00000100:00100000:1.0:1789760754.558920:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 441 00000100:00100000:1.0:1789760754.558929:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e67800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534075008:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760754.558966:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760754.559115:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e67800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534075008:12345-0@lo:41:osp-pre-1-0.0 Request processed in 185us (311us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760754.559122:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 441 00000100:00100000:3.0:1789760754.559313:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887e64380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534075008:0@lo:41:osp-pre-1-0.0 00000001:00080000:2.0:1789760754.584841:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972732 is committed 00002000:00010000:1.0:1789760754.584945:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22167552 -> 22171648 00010000:00010000:1.0:1789760754.584959:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.584971:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.584990:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.584997:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760754.585002:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.585010:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b0b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.585034:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b0b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.585041:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22167552->22171647] (req 22167552->22171647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.585133:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508375296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 75201us (75477us total) trans 4294972732 rc 0/0 00000100:00100000:1.0:1789760754.585143:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5444 00000100:00100000:1.0:1789760754.585154:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733573115395 time 733614624926 state 0x2 ref 0 00000100:00100000:2.0:1789760754.591702:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760754.591793:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508375424 00000100:00100000:3.0:1789760754.591813:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508375424 00000100:00100000:3.0:1789760754.591867:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733573115395 time 733614624926 state 0x0 ref 1 00000100:00100000:3.0:1789760754.591882:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733614885342 time 733614624926 state 0x1 ref 1 00000100:00100000:3.0:1789760754.592137:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733614885342 time 733696845908 state 0x0 ref 1 00000100:00100000:3.0:1789760754.592151:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508375424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760754.592197:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22171648, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:3.0:1789760754.592206:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b12 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.592222:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.592226:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.592232:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.592246:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.634969:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972733 is committed 00002000:00010000:0.0:1789760754.635062:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22171648 -> 22175744 00010000:00010000:0.0:1789760754.635077:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.635090:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b12 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.635099:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.635107:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760754.635111:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.635121:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b12 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.635141:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b12 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.635149:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b12 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22171648->22175743] (req 22171648->22175743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760754.635282:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508375424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43128us (43583us total) trans 4294972733 rc 0/0 00000100:00100000:0.0:1789760754.635336:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5445 00000100:00100000:0.0:1789760754.635346:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733614885342 time 733696845908 state 0x2 ref 0 00000100:00100000:2.0:1789760754.639583:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760754.639660:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508375552 00000100:00100000:1.0:1789760754.639671:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508375552 00000100:00100000:1.0:1789760754.639729:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733614885342 time 733696845908 state 0x0 ref 1 00000100:00100000:1.0:1789760754.639746:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733697106324 time 733696845908 state 0x1 ref 1 00000100:00100000:1.0:1789760754.639779:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733697106324 time 733744489690 state 0x0 ref 1 00000100:00100000:1.0:1789760754.639792:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508375552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760754.639843:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22175744, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:1.0:1789760754.639853:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b19 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.639868:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760754.639874:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.639880:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.639894:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760754.670658:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972734 is committed 00002000:00010000:1.0:1789760754.670792:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22175744 -> 22179840 00010000:00010000:1.0:1789760754.670807:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.670820:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.670830:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.670837:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760754.670842:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.670851:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b19 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.670870:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b19 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.670883:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b19 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22175744->22179839] (req 22175744->22179839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.671020:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508375552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31224us (31441us total) trans 4294972734 rc 0/0 00000100:00100000:1.0:1789760754.671031:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5446 00000100:00100000:1.0:1789760754.671040:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733697106324 time 733744489690 state 0x2 ref 0 00000100:00100000:2.0:1789760754.674743:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760754.674847:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508375680 00000100:00100000:1.0:1789760754.674857:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508375680 00000100:00100000:1.0:1789760754.674922:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733697106324 time 733744489690 state 0x0 ref 1 00000100:00100000:1.0:1789760754.674937:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733744750106 time 733744489690 state 0x1 ref 1 00000100:00100000:1.0:1789760754.674964:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733744750106 time 733779674668 state 0x0 ref 1 00000100:00100000:1.0:1789760754.674985:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508375680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760754.675032:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22179840, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:1.0:1789760754.675040:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b20 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.675056:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760754.675061:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.675067:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.675086:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.702311:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972735 is committed 00002000:00010000:1.0:1789760754.702484:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22179840 -> 22183936 00010000:00010000:1.0:1789760754.702500:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.702512:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b20 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.702521:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.702529:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760754.702534:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.702543:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.702563:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b20 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.702571:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b20 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22179840->22183935] (req 22179840->22183935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760754.702671:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508375680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27691us (27932us total) trans 4294972735 rc 0/0 00000100:00100000:1.0:1789760754.702680:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5447 00000100:00100000:1.0:1789760754.702692:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733744750106 time 733779674668 state 0x2 ref 0 00000100:00100000:2.0:1789760754.706648:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760754.706739:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508375808 00000100:00100000:0.0:1789760754.706749:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508375808 00000100:00100000:0.0:1789760754.706807:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733744750106 time 733779674668 state 0x0 ref 1 00000100:00100000:0.0:1789760754.706821:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733779935084 time 733779674668 state 0x1 ref 1 00000100:00100000:0.0:1789760754.706852:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733779935084 time 733811562693 state 0x0 ref 1 00000100:00100000:0.0:1789760754.706862:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508375808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760754.706921:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22183936, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:0.0:1789760754.706930:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55b27 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.706946:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760754.706950:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760754.706956:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55b27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760754.706970:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55b27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760754.740184:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972736 is committed 00002000:00010000:2.0:1789760754.740561:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22183936 -> 22188032 00010000:00010000:2.0:1789760754.740576:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55b27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.740587:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55b27 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.740595:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55b27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.740602:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760754.740607:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55b27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.740616:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55b27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.740634:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55b27 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.740645:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55b27 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22183936->22188031] (req 22183936->22188031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.740774:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508375808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33908us (34130us total) trans 4294972736 rc 0/0 00000100:00100000:2.0:1789760754.740785:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5448 00000100:00100000:2.0:1789760754.740794:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733779935084 time 733811562693 state 0x2 ref 0 00000100:00100000:2.0:1789760754.745083:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760754.745159:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508375936 00000100:00100000:1.0:1789760754.745168:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508375936 00000100:00100000:1.0:1789760754.745217:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733779935084 time 733811562693 state 0x0 ref 1 00000100:00100000:1.0:1789760754.745232:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733811823109 time 733811562693 state 0x1 ref 1 00000100:00100000:1.0:1789760754.745256:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733811823109 time 733849967189 state 0x0 ref 1 00000100:00100000:1.0:1789760754.745267:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508375936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760754.745346:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22188032, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:1.0:1789760754.745355:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.745370:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760754.745374:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760754.745380:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760754.745393:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.770846:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972737 is committed 00002000:00010000:3.0:1789760754.770961:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22188032 -> 22192128 00010000:00010000:3.0:1789760754.770986:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.771005:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.771014:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.771022:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760754.771027:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.771037:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.771057:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.771065:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22188032->22192127] (req 22188032->22192127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.771163:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508375936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25892us (26083us total) trans 4294972737 rc 0/0 00000100:00100000:3.0:1789760754.771172:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5449 00000100:00100000:3.0:1789760754.771181:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733811823109 time 733849967189 state 0x2 ref 0 00000100:00100000:0.0:1789760754.776213:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760754.776389:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508376064 00000100:00100000:1.0:1789760754.776399:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508376064 00000100:00100000:2.0:1789760754.776501:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733811823109 time 733849967189 state 0x0 ref 1 00000100:00100000:2.0:1789760754.776518:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733850227605 time 733849967189 state 0x1 ref 1 00000100:00100000:2.0:1789760754.776543:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733850227605 time 733881253579 state 0x0 ref 1 00000100:00100000:2.0:1789760754.776556:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508376064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760754.776606:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22192128, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:2.0:1789760754.776627:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55b35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.776643:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760754.776647:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.776652:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55b35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.776671:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55b35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.813385:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972738 is committed 00002000:00010000:3.0:1789760754.813660:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22192128 -> 22196224 00010000:00010000:3.0:1789760754.813678:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55b35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.813691:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55b35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.813699:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55b35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.813706:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760754.813712:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55b35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.813721:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55b35 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.813819:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55b35 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.813828:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55b35 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22192128->22196223] (req 22192128->22196223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.813927:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508376064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37368us (37717us total) trans 4294972738 rc 0/0 00000100:00100000:3.0:1789760754.813938:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5450 00000100:00100000:3.0:1789760754.814080:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733850227605 time 733881253579 state 0x2 ref 0 00000100:00100000:0.0:1789760754.821111:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760754.821215:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508376192 00000100:00100000:2.0:1789760754.821226:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508376192 00000100:00100000:2.0:1789760754.821280:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733850227605 time 733881253579 state 0x0 ref 1 00000100:00100000:2.0:1789760754.821335:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733881513995 time 733881253579 state 0x1 ref 1 00000100:00100000:2.0:1789760754.821408:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733881513995 time 733926117258 state 0x0 ref 1 00000100:00100000:2.0:1789760754.821421:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508376192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760754.821472:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22196224, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:2.0:1789760754.821481:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b3c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.821497:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760754.821502:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.821508:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.821523:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.864330:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972739 is committed 00002000:00010000:2.0:1789760754.864586:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22196224 -> 22200320 00010000:00010000:2.0:1789760754.864602:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.864614:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b3c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.864622:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.864630:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760754.864635:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.864645:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b3c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.864664:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b3c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.864673:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b3c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22196224->22200319] (req 22196224->22200319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.864985:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508376192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43554us (43872us total) trans 4294972739 rc 0/0 00000100:00100000:2.0:1789760754.864997:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5451 00000100:00100000:2.0:1789760754.865006:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733881513995 time 733926117258 state 0x2 ref 0 00000100:00100000:2.0:1789760754.871144:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760754.871565:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508376320 00000100:00100000:3.0:1789760754.871574:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508376320 00000100:00100000:3.0:1789760754.871626:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733881513995 time 733926117258 state 0x0 ref 1 00000100:00100000:3.0:1789760754.871643:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733926377674 time 733926117258 state 0x1 ref 1 00000100:00100000:3.0:1789760754.871711:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733926377674 time 733976420899 state 0x0 ref 1 00000100:00100000:3.0:1789760754.871725:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508376320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760754.871782:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22200320, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:3.0:1789760754.871791:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.871808:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760754.871812:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.871819:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.871833:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.898566:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22200320 -> 22204416 00010000:00010000:3.0:1789760754.898587:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.898599:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.898608:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760754.898615:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972740 is committed 00010000:00010000:3.0:1789760754.898615:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760754.898619:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.898627:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.898649:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b43 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.898656:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22200320->22204415] (req 22200320->22204415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.898750:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508376320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27022us (27610us total) trans 4294972740 rc 0/0 00000100:00100000:3.0:1789760754.898760:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5452 00000100:00100000:3.0:1789760754.898771:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733926377674 time 733976420899 state 0x2 ref 0 00000100:00100000:0.0:1789760754.902103:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760754.902210:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508376448 00000100:00100000:3.0:1789760754.902221:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508376448 00000100:00100000:3.0:1789760754.902282:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733926377674 time 733976420899 state 0x0 ref 1 00000100:00100000:3.0:1789760754.902329:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733976681315 time 733976420899 state 0x1 ref 1 00000100:00100000:3.0:1789760754.902362:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733976681315 time 734007072334 state 0x0 ref 1 00000100:00100000:3.0:1789760754.902375:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508376448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760754.902429:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22204416, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:3.0:1789760754.902439:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55b4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.902455:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760754.902460:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.902467:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55b4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.902482:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55b4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760754.934484:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972741 is committed 00002000:00010000:2.0:1789760754.934721:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22204416 -> 22208512 00010000:00010000:2.0:1789760754.934737:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55b4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.934747:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55b4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.934757:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55b4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.934764:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760754.934769:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55b4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.934778:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55b4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.934797:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55b4a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.934806:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe55b4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22204416->22208511] (req 22204416->22208511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760754.934902:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508376448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32524us (32806us total) trans 4294972741 rc 0/0 00000100:00100000:2.0:1789760754.934916:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5453 00000100:00100000:2.0:1789760754.934925:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733976681315 time 734007072334 state 0x2 ref 0 00000100:00100000:0.0:1789760754.947086:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760754.947426:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508376576 00000100:00100000:2.0:1789760754.947445:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508376576 00000100:00100000:2.0:1789760754.947507:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 733976681315 time 734007072334 state 0x0 ref 1 00000100:00100000:2.0:1789760754.947524:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734007332750 time 734007072334 state 0x1 ref 1 00000100:00100000:2.0:1789760754.947598:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734007332750 time 734052307699 state 0x0 ref 1 00000100:00100000:2.0:1789760754.947611:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508376576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760754.947662:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22208512, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:2.0:1789760754.947671:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.947688:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760754.947692:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760754.947698:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760754.947712:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.015208:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972742 is committed 00002000:00010000:2.0:1789760755.015585:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22208512 -> 22212608 00010000:00010000:2.0:1789760755.015601:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.015614:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.015622:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.015630:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760755.015635:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.015643:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.015662:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b51 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.015669:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22208512->22212607] (req 22208512->22212607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.015963:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508376576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 68349us (68881us total) trans 4294972742 rc 0/0 00000100:00100000:2.0:1789760755.015984:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5454 00000100:00100000:2.0:1789760755.015994:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734007332750 time 734052307699 state 0x2 ref 0 00000100:00100000:2.0:1789760755.020189:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760755.021464:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508376704 00000100:00100000:3.0:1789760755.021481:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508376704 00000100:00100000:3.0:1789760755.021578:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734007332750 time 734052307699 state 0x0 ref 1 00000100:00100000:3.0:1789760755.021597:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734052568115 time 734052307699 state 0x1 ref 1 00000100:00100000:3.0:1789760755.021621:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734052568115 time 734126332504 state 0x0 ref 1 00000100:00100000:3.0:1789760755.021636:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508376704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760755.021702:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22212608, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:3.0:1789760755.021712:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.021730:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.021735:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.021743:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.021758:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760755.022127:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760755.062872:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972743 is committed 00002000:00010000:3.0:1789760755.064442:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22212608 -> 22216704 00010000:00010000:3.0:1789760755.064468:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.065931:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.065959:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.065969:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760755.065983:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.065996:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.066024:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b58 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.066033:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22212608->22216703] (req 22212608->22216703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.066179:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508376704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44538us (45997us total) trans 4294972743 rc 0/0 00000100:00100000:3.0:1789760755.066193:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5455 00000100:00100000:3.0:1789760755.066206:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734052568115 time 734126332504 state 0x2 ref 0 00000100:00100000:3.0:1789760755.069179:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760755.071947:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508376832 00000100:00100000:2.0:1789760755.071979:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508376832 00000100:00100000:2.0:1789760755.072130:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734052568115 time 734126332504 state 0x0 ref 1 00000100:00100000:2.0:1789760755.072153:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734126592920 time 734126332504 state 0x1 ref 1 00000100:00100000:2.0:1789760755.072198:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734126592920 time 734176907625 state 0x0 ref 1 00000100:00100000:2.0:1789760755.072214:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508376832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760755.072304:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22216704, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:2.0:1789760755.072315:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b5f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.072335:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760755.072341:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.072349:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.072365:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.120209:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22216704 -> 22220800 00010000:00010000:3.0:1789760755.120230:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.120244:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.120253:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.120260:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760755.120265:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.120279:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b5f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.120472:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972744 is committed 00010000:00010000:3.0:1789760755.120534:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b5f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.120543:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55b5f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22216704->22220799] (req 22216704->22220799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.120668:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508376832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48452us (51499us total) trans 4294972744 rc 0/0 00000100:00100000:3.0:1789760755.120679:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5456 00000100:00100000:3.0:1789760755.120689:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734126592920 time 734176907625 state 0x2 ref 0 00000100:00100000:0.0:1789760755.131513:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760755.132028:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508376960 00000100:00100000:2.0:1789760755.132155:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508376960 00000100:00100000:2.0:1789760755.132227:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734126592920 time 734176907625 state 0x0 ref 1 00000100:00100000:2.0:1789760755.132244:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734177168041 time 734176907625 state 0x1 ref 1 00000100:00100000:2.0:1789760755.132276:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734177168041 time 734236986813 state 0x0 ref 1 00000100:00100000:2.0:1789760755.132789:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508376960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760755.132842:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22220800, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:2.0:1789760755.132852:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55b66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.132867:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760755.132872:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.132878:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55b66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.132893:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55b66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.187235:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972745 is committed 00002000:00010000:0.0:1789760755.187469:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22220800 -> 22224896 00010000:00010000:0.0:1789760755.187485:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55b66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.187498:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55b66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.187507:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55b66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.187515:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760755.187519:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55b66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.187528:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55b66 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.187552:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55b66 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.187560:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe55b66 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22220800->22224895] (req 22220800->22224895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.187664:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508376960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55372us (56155us total) trans 4294972745 rc 0/0 00000100:00100000:0.0:1789760755.187675:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5457 00000100:00100000:0.0:1789760755.187684:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734177168041 time 734236986813 state 0x2 ref 0 00000100:00100000:2.0:1789760755.197397:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760755.197483:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508377088 00000100:00100000:3.0:1789760755.197493:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508377088 00000100:00100000:3.0:1789760755.197544:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734177168041 time 734236986813 state 0x0 ref 1 00000100:00100000:3.0:1789760755.197558:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734237247229 time 734236986813 state 0x1 ref 1 00000100:00100000:3.0:1789760755.197591:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734237247229 time 734302301997 state 0x0 ref 1 00000100:00100000:3.0:1789760755.197603:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508377088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760755.197651:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22224896, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:3.0:1789760755.197661:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.197675:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.197680:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.197687:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.197700:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.260825:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22224896 -> 22228992 00010000:00010000:3.0:1789760755.260842:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.260855:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.260865:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.260872:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760755.260877:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.260886:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.260910:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.260918:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55b6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22224896->22228991] (req 22224896->22228991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.261014:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508377088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 63408us (63623us total) trans 4294972746 rc 0/0 00000100:00100000:3.0:1789760755.261025:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5458 00000100:00100000:3.0:1789760755.261034:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734237247229 time 734302301997 state 0x2 ref 0 00000001:00080000:0.0:1789760755.261105:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972746 is committed 00000100:00100000:3.0:1789760755.265516:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760755.265585:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508377216 00000100:00100000:2.0:1789760755.265599:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508377216 00000100:00100000:2.0:1789760755.265660:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734237247229 time 734302301997 state 0x0 ref 1 00000100:00100000:2.0:1789760755.265674:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734302562413 time 734302301997 state 0x1 ref 1 00000100:00100000:2.0:1789760755.265704:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734302562413 time 734370414919 state 0x0 ref 1 00000100:00100000:2.0:1789760755.265716:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508377216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760755.265762:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22228992, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:2.0:1789760755.265770:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.265784:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760755.265788:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.265794:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.265808:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760755.311650:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972747 is committed 00002000:00010000:2.0:1789760755.311738:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22228992 -> 22233088 00010000:00010000:2.0:1789760755.311758:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.311774:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.311784:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.311792:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760755.311798:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.311815:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.311841:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b74 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.311850:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b74 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22228992->22233087] (req 22228992->22233087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.312216:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508377216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46244us (46469us total) trans 4294972747 rc 0/0 00000100:00100000:2.0:1789760755.312229:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5459 00000100:00100000:2.0:1789760755.312242:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734302562413 time 734370414919 state 0x2 ref 0 00000100:00100000:2.0:1789760755.315361:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760755.315513:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508377344 00000100:00100000:0.0:1789760755.315525:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508377344 00000100:00100000:0.0:1789760755.315611:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734302562413 time 734370414919 state 0x0 ref 1 00000100:00100000:0.0:1789760755.315632:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734370675335 time 734370414919 state 0x1 ref 1 00000100:00100000:0.0:1789760755.315724:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734370675335 time 734420433136 state 0x0 ref 1 00000100:00100000:0.0:1789760755.315739:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508377344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760755.315806:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22233088, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:0.0:1789760755.315818:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.315843:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760755.315848:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.315855:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.315870:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.357457:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972748 is committed 00002000:00010000:1.0:1789760755.357745:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22233088 -> 22237184 00010000:00010000:1.0:1789760755.357766:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.357781:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.357791:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.357809:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760755.357814:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.357825:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.357851:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b7b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.357860:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55b7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22233088->22237183] (req 22233088->22237183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.358020:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508377344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42275us (42667us total) trans 4294972748 rc 0/0 00000100:00100000:1.0:1789760755.358032:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5460 00000100:00100000:1.0:1789760755.358046:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734370675335 time 734420433136 state 0x2 ref 0 00000100:00100000:2.0:1789760755.364221:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760755.364349:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508377472 00000100:00100000:3.0:1789760755.364362:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508377472 00000100:00100000:3.0:1789760755.364437:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734370675335 time 734420433136 state 0x0 ref 1 00000100:00100000:3.0:1789760755.364456:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734420693552 time 734420433136 state 0x1 ref 1 00000100:00100000:3.0:1789760755.364490:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734420693552 time 734469200494 state 0x0 ref 1 00000100:00100000:3.0:1789760755.364504:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508377472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760755.364737:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22237184, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:3.0:1789760755.364748:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b82 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.364773:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.364779:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.364786:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.364802:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.411769:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972749 is committed 00002000:00010000:2.0:1789760755.412051:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22237184 -> 22241280 00010000:00010000:2.0:1789760755.412071:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.412085:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b82 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.412094:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.412108:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760755.412114:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.412124:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.412154:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b82 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.412173:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe55b82 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22237184->22241279] (req 22237184->22241279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.412349:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508377472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47839us (48138us total) trans 4294972749 rc 0/0 00000100:00100000:2.0:1789760755.412361:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5461 00000100:00100000:2.0:1789760755.412374:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734420693552 time 734469200494 state 0x2 ref 0 00000100:00100000:3.0:1789760755.416671:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760755.416901:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508377600 00000100:00100000:2.0:1789760755.416916:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508377600 00000100:00100000:2.0:1789760755.417125:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734420693552 time 734469200494 state 0x0 ref 1 00000100:00100000:2.0:1789760755.417144:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734469460910 time 734469200494 state 0x1 ref 1 00000100:00100000:2.0:1789760755.417281:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734469460910 time 734521989753 state 0x0 ref 1 00000100:00100000:2.0:1789760755.417339:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508377600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760755.417406:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22241280, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:2.0:1789760755.417417:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b89 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.417437:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760755.417441:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.417449:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.417477:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.456420:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760755.465046:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972750 is committed 00002000:00010000:2.0:1789760755.465241:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22241280 -> 22245376 00010000:00010000:2.0:1789760755.465263:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.465280:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b89 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.465311:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.465319:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760755.465325:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.465337:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b89 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.465374:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b89 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.465384:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55b89 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22241280->22245375] (req 22241280->22245375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.465518:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508377600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48177us (48855us total) trans 4294972750 rc 0/0 00000100:00100000:2.0:1789760755.465530:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5462 00000100:00100000:2.0:1789760755.465543:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734469460910 time 734521989753 state 0x2 ref 0 00000100:00100000:3.0:1789760755.469092:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760755.469218:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508377728 00000100:00100000:1.0:1789760755.469231:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508377728 00000100:00100000:1.0:1789760755.469345:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734469460910 time 734521989753 state 0x0 ref 1 00000100:00100000:1.0:1789760755.469364:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734522250169 time 734521989753 state 0x1 ref 1 00000100:00100000:1.0:1789760755.469396:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734522250169 time 734574106538 state 0x0 ref 1 00000100:00100000:1.0:1789760755.469410:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508377728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760755.469474:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22245376, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0xb007 00010000:00010000:1.0:1789760755.469484:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.469503:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760755.469508:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.469514:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.469530:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.533593:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972751 is committed 00002000:00010000:1.0:1789760755.533701:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22245376 -> 22249472 00002000:00010000:1.0:1789760755.533707:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760755 -> 1789760756 00002000:00010000:1.0:1789760755.533710:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760755 -> 1789760756 00010000:00010000:1.0:1789760755.533729:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.533746:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.533756:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.533765:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760755.533772:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.533783:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.533807:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b90 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.533817:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55b90 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22245376->22249471] (req 22245376->22249471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.534113:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508377728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 64698us (65028us total) trans 4294972751 rc 0/0 00000100:00100000:1.0:1789760755.534126:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5463 00000100:00100000:1.0:1789760755.534140:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734522250169 time 734574106538 state 0x2 ref 0 00000100:00100000:2.0:1789760755.537837:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760755.538085:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508377856 00000100:00100000:3.0:1789760755.538100:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508377856 00000100:00100000:3.0:1789760755.538254:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734522250169 time 734574106538 state 0x0 ref 1 00000100:00100000:3.0:1789760755.538275:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734574366954 time 734574106538 state 0x1 ref 1 00000100:00100000:1.0:1789760755.538549:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734574366954 time 734643255417 state 0x0 ref 1 00000100:00100000:1.0:1789760755.538569:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508377856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760755.538642:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22249472, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:1.0:1789760755.538652:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b97 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.538671:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760755.538676:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.538684:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.538701:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.572975:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972752 is committed 00002000:00010000:0.0:1789760755.573095:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22249472 -> 22253568 00010000:00010000:0.0:1789760755.573116:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.573131:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.573142:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.573150:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760755.573166:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.573178:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.573208:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b97 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.573220:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55b97 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22249472->22253567] (req 22249472->22253567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760755.573383:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508377856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34813us (35558us total) trans 4294972752 rc 0/0 00000100:00100000:0.0:1789760755.573397:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5464 00000100:00100000:0.0:1789760755.573410:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734574366954 time 734643255417 state 0x2 ref 0 00000100:00100000:3.0:1789760755.576862:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760755.577108:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508377984 00000100:00100000:0.0:1789760755.577123:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508377984 00000100:00100000:0.0:1789760755.577204:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734574366954 time 734643255417 state 0x0 ref 1 00000100:00100000:0.0:1789760755.577222:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734643515833 time 734643255417 state 0x1 ref 1 00000100:00100000:0.0:1789760755.577248:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734643515833 time 734681958093 state 0x0 ref 1 00000100:00100000:0.0:1789760755.577262:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508377984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760755.577361:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22253568, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:0.0:1789760755.577372:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55b9e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.577392:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760755.577397:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.577405:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55b9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.577427:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55b9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.610766:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972753 is committed 00002000:00010000:1.0:1789760755.610860:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22253568 -> 22257664 00010000:00010000:1.0:1789760755.610877:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55b9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.610938:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55b9e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.610946:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55b9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.610951:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760755.610956:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55b9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.610962:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55b9e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.610997:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55b9e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.611005:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55b9e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22253568->22257663] (req 22253568->22257663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.611177:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508377984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33910us (34321us total) trans 4294972753 rc 0/0 00000100:00100000:1.0:1789760755.611189:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5465 00000100:00100000:1.0:1789760755.611201:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734643515833 time 734681958093 state 0x2 ref 0 00000100:00100000:0.0:1789760755.620672:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760755.620800:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508378112 00000100:00100000:1.0:1789760755.620813:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508378112 00000100:00100000:1.0:1789760755.620885:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734643515833 time 734681958093 state 0x0 ref 1 00000100:00100000:1.0:1789760755.620905:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734682218509 time 734681958093 state 0x1 ref 1 00000100:00100000:1.0:1789760755.620944:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734682218509 time 734725654359 state 0x0 ref 1 00000100:00100000:1.0:1789760755.620959:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508378112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760755.621036:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22257664, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:1.0:1789760755.621048:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ba5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.621067:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760755.621072:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.621079:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ba5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.621096:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ba5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.664858:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22257664 -> 22261760 00010000:00010000:3.0:1789760755.664897:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ba5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.664914:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ba5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.664925:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ba5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.664934:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760755.664940:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ba5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.664950:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ba5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.664981:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ba5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.664990:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ba5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22257664->22261759] (req 22257664->22261759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.665226:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508378112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44260us (44561us total) trans 4294972754 rc 0/0 00000100:00100000:3.0:1789760755.665238:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5466 00000100:00100000:3.0:1789760755.665250:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734682218509 time 734725654359 state 0x2 ref 0 00000001:00080000:2.0:1789760755.666196:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972754 is committed 00000100:00100000:2.0:1789760755.672839:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760755.674549:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508378240 00000100:00100000:0.0:1789760755.674568:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508378240 00000100:00100000:0.0:1789760755.674651:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734682218509 time 734725654359 state 0x0 ref 1 00000100:00100000:0.0:1789760755.674671:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734725914775 time 734725654359 state 0x1 ref 1 00000100:00100000:0.0:1789760755.674706:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734725914775 time 734779416233 state 0x0 ref 1 00000100:00100000:0.0:1789760755.674721:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508378240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760755.674787:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22261760, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:0.0:1789760755.674798:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55bac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.674818:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760755.674823:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760755.674831:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55bac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760755.674848:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55bac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.709493:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972755 is committed 00002000:00010000:2.0:1789760755.709688:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22261760 -> 22265856 00010000:00010000:2.0:1789760755.709710:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55bac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.709725:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55bac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.709734:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55bac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.709743:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760755.709748:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55bac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.709759:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55bac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.709786:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55bac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.709795:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55bac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22261760->22265855] (req 22261760->22265855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.710034:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508378240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35235us (37129us total) trans 4294972755 rc 0/0 00000100:00100000:2.0:1789760755.710047:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5467 00000100:00100000:2.0:1789760755.710059:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734725914775 time 734779416233 state 0x2 ref 0 00000100:00100000:0.0:1789760755.715231:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760755.715624:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508378368 00000100:00100000:1.0:1789760755.715638:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508378368 00000100:00100000:1.0:1789760755.715718:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734725914775 time 734779416233 state 0x0 ref 1 00000100:00100000:1.0:1789760755.715737:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734779676649 time 734779416233 state 0x1 ref 1 00000100:00100000:1.0:1789760755.715808:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734779676649 time 734820517868 state 0x0 ref 1 00000100:00100000:1.0:1789760755.715822:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508378368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760755.715897:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22265856, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:1.0:1789760755.715910:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55bb3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.715930:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760755.715936:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760755.715943:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55bb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.716149:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55bb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760755.748936:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22265856 -> 22269952 00010000:00010000:2.0:1789760755.749041:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55bb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.749059:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55bb3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.749076:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55bb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.749084:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760755.749089:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55bb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.749099:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55bb3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.749125:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55bb3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.749134:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55bb3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22265856->22269951] (req 22265856->22269951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.749380:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972756 is committed 00000100:00100000:2.0:1789760755.749641:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508378368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33813us (34417us total) trans 4294972756 rc 0/0 00000100:00100000:2.0:1789760755.749654:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5468 00000100:00100000:2.0:1789760755.749666:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734779676649 time 734820517868 state 0x2 ref 0 00000100:00100000:2.0:1789760755.759283:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760755.759424:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508378496 00000100:00100000:3.0:1789760755.759439:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508378496 00000100:00100000:3.0:1789760755.759575:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734779676649 time 734820517868 state 0x0 ref 1 00000100:00100000:3.0:1789760755.759596:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734820778284 time 734820517868 state 0x1 ref 1 00000100:00100000:3.0:1789760755.760552:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734820778284 time 734865254380 state 0x0 ref 1 00000100:00100000:3.0:1789760755.760579:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508378496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760755.762281:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22269952, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:3.0:1789760755.762335:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55bba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.762359:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760755.762366:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.762374:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55bba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.762392:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55bba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.778846:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760755.779158:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760755.779199:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760755.780332:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760755.780365:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760755.780403:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508378752 00000100:00100000:3.0:1789760755.780418:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508378752 00000100:00100000:3.0:1789760755.780511:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508379008 00000100:00100000:3.0:1789760755.780516:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508379008 00000100:00100000:3.0:1789760755.780545:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508378880 00000100:00100000:3.0:1789760755.780548:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508378880 00000100:00100000:3.0:1789760755.780564:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508379136 00000100:00100000:3.0:1789760755.780567:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508379136 00000100:00100000:3.0:1789760755.780589:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 919 00000100:00100000:3.0:1789760755.780602:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508378752:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:3.0:1789760755.780719:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508378752:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 119us (1889us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760755.780728:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 919 00000100:00100000:3.0:1789760755.780748:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 920 00000100:00100000:3.0:1789760755.780755:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508379008:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:3.0:1789760755.780781:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508379008:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 27us (1626us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760755.780788:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 920 00000100:00100000:3.0:1789760755.780799:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 921 00000100:00100000:3.0:1789760755.780805:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508378880:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:3.0:1789760755.780828:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508378880:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 23us (507us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760755.780835:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 921 00000100:00100000:3.0:1789760755.780845:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 922 00000100:00100000:3.0:1789760755.780851:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508379136:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:3.0:1789760755.780870:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508379136:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 20us (504us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760755.780876:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 922 00000100:00080000:1.0:1789760755.781112:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760755.781121:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760755.781261:0:6460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508378624 00000100:00100000:0.0:1789760755.781269:0:6460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508378624 00000100:00100000:1.0:1789760755.781473:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 477 00000100:00100000:1.0:1789760755.781559:0:6460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cacc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2408:x1876699508378624:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760755.781636:0:6460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cacc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2408:x1876699508378624:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 151us (2434us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760755.781648:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 477 00000001:00080000:3.0:1789760755.882496:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972757 is committed 00002000:00010000:2.0:1789760755.882650:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22269952 -> 22274048 00010000:00010000:2.0:1789760755.882666:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55bba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.882682:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55bba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.882690:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55bba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.882697:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760755.882701:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55bba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.882710:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55bba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.882732:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55bba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.882739:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55bba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22269952->22274047] (req 22269952->22274047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760755.882845:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508378496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 122270us (123572us total) trans 4294972757 rc 0/0 00000100:00100000:2.0:1789760755.882857:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5469 00000100:00100000:2.0:1789760755.882866:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734820778284 time 734865254380 state 0x2 ref 0 00000100:00100000:1.0:1789760755.889774:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760755.889898:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508379264 00000100:00100000:2.0:1789760755.889910:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508379264 00000100:00100000:2.0:1789760755.889982:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734820778284 time 734865254380 state 0x0 ref 1 00000100:00100000:2.0:1789760755.889998:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734865514796 time 734865254380 state 0x1 ref 1 00000100:00100000:2.0:1789760755.890082:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734865514796 time 734994790450 state 0x0 ref 1 00000100:00100000:2.0:1789760755.890095:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508379264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760755.890145:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22274048, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:2.0:1789760755.890155:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55bc1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.890170:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760755.890174:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.890180:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55bc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.890194:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55bc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760755.933800:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972758 is committed 00002000:00010000:3.0:1789760755.934116:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22274048 -> 22278144 00010000:00010000:3.0:1789760755.934131:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55bc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.934142:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55bc1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.934149:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55bc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.934155:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760755.934159:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55bc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.934168:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55bc1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.934189:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55bc1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.934197:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe55bc1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22274048->22278143] (req 22274048->22278143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.934448:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508379264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44350us (44679us total) trans 4294972758 rc 0/0 00000100:00100000:3.0:1789760755.934459:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5470 00000100:00100000:3.0:1789760755.934469:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734865514796 time 734994790450 state 0x2 ref 0 00000100:00100000:0.0:1789760755.937097:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760755.937189:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508379392 00000100:00100000:2.0:1789760755.937199:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508379392 00000100:00100000:2.0:1789760755.937256:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734865514796 time 734994790450 state 0x0 ref 1 00000100:00100000:2.0:1789760755.937271:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734995050866 time 734994790450 state 0x1 ref 1 00000100:00100000:2.0:1789760755.937336:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734995050866 time 735042045820 state 0x0 ref 1 00000100:00100000:2.0:1789760755.937353:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508379392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760755.937401:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22278144, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:2.0:1789760755.937410:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55bc8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.937425:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760755.937430:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.937436:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55bc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.937450:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55bc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.980500:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972759 is committed 00002000:00010000:1.0:1789760755.980672:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22278144 -> 22282240 00010000:00010000:1.0:1789760755.980687:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55bc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.980700:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55bc8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.980716:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55bc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.980724:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760755.980729:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55bc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.980739:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55bc8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.980759:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55bc8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760755.980768:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe55bc8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22278144->22282239] (req 22278144->22282239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760755.980863:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508379392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43507us (43844us total) trans 4294972759 rc 0/0 00000100:00100000:1.0:1789760755.980874:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5471 00000100:00100000:1.0:1789760755.981722:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734995050866 time 735042045820 state 0x2 ref 0 00000100:00100000:1.0:1789760755.984572:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760755.984845:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508379520 00000100:00100000:2.0:1789760755.984855:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508379520 00000100:00100000:2.0:1789760755.984919:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 734995050866 time 735042045820 state 0x0 ref 1 00000100:00100000:2.0:1789760755.984935:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735042306236 time 735042045820 state 0x1 ref 1 00000100:00100000:2.0:1789760755.984975:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735042306236 time 735089676465 state 0x0 ref 1 00000100:00100000:2.0:1789760755.984987:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508379520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760755.985036:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22282240, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:2.0:1789760755.985044:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55bcf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.985064:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760755.985068:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760755.985074:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55bcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760755.985087:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55bcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.026619:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22282240 -> 22286336 00010000:00010000:1.0:1789760756.026640:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55bcf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.026659:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55bcf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.026668:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55bcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.026675:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760756.026680:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55bcf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.026688:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55bcf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.026708:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55bcf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.026716:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe55bcf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22282240->22286335] (req 22282240->22286335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.026810:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508379520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41819us (42242us total) trans 4294972760 rc 0/0 00000100:00100000:1.0:1789760756.026821:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5472 00000100:00100000:1.0:1789760756.026831:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735042306236 time 735089676465 state 0x2 ref 0 00000001:00080000:3.0:1789760756.030165:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972760 is committed 00000100:00100000:0.0:1789760756.030413:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760756.030826:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508379648 00000100:00100000:3.0:1789760756.030836:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508379648 00000100:00100000:3.0:1789760756.030956:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735042306236 time 735089676465 state 0x0 ref 1 00000100:00100000:3.0:1789760756.030981:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735089936881 time 735089676465 state 0x1 ref 1 00000100:00100000:3.0:1789760756.031044:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735089936881 time 735135753895 state 0x0 ref 1 00000100:00100000:3.0:1789760756.031058:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508379648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760756.031102:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22286336, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:3.0:1789760756.031111:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55bd6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.031125:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760756.031130:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.031137:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55bd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.031150:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55bd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.059275:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22286336 -> 22290432 00010000:00010000:1.0:1789760756.059326:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55bd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.059339:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55bd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.059349:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55bd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.059358:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760756.059363:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55bd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.059373:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55bd6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.059395:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55bd6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.059404:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe55bd6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22286336->22290431] (req 22286336->22290431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.059508:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508379648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28446us (29094us total) trans 4294972761 rc 0/0 00000100:00100000:1.0:1789760756.059519:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5473 00000100:00100000:1.0:1789760756.059530:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735089936881 time 735135753895 state 0x2 ref 0 00000001:00080000:3.0:1789760756.062424:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972761 is committed 00000100:00100000:1.0:1789760756.062809:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760756.063013:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508379776 00000100:00100000:3.0:1789760756.063022:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508379776 00000100:00100000:3.0:1789760756.063076:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735089936881 time 735135753895 state 0x0 ref 1 00000100:00100000:3.0:1789760756.063092:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735136014311 time 735135753895 state 0x1 ref 1 00000100:00100000:3.0:1789760756.063124:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735136014311 time 735167833843 state 0x0 ref 1 00000100:00100000:3.0:1789760756.063136:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508379776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760756.063185:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22290432, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:3.0:1789760756.063194:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55bdd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.063208:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760756.063213:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.063219:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55bdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.063233:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55bdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.136772:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972762 is committed 00002000:00010000:1.0:1789760756.138492:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22290432 -> 22294528 00010000:00010000:1.0:1789760756.138510:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55bdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.138524:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55bdd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.138534:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55bdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.138542:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760756.138548:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55bdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.138557:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55bdd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.138578:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55bdd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.138587:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55bdd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22290432->22294527] (req 22290432->22294527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.138678:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508379776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 75538us (75869us total) trans 4294972762 rc 0/0 00000100:00100000:1.0:1789760756.138694:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5474 00000100:00100000:1.0:1789760756.138703:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735136014311 time 735167833843 state 0x2 ref 0 00000100:00100000:2.0:1789760756.142421:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760756.142559:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508379904 00000100:00100000:1.0:1789760756.142569:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508379904 00000100:00100000:1.0:1789760756.142623:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735136014311 time 735167833843 state 0x0 ref 1 00000100:00100000:1.0:1789760756.142640:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735168094259 time 735167833843 state 0x1 ref 1 00000100:00100000:1.0:1789760756.142677:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735168094259 time 735247386714 state 0x0 ref 1 00000100:00100000:1.0:1789760756.142689:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508379904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760756.142737:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22294528, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:1.0:1789760756.142747:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55be4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.142762:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760756.142767:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.142774:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55be4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.142788:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55be4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.265915:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972763 is committed 00002000:00010000:1.0:1789760756.267054:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22294528 -> 22298624 00010000:00010000:1.0:1789760756.267073:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55be4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.267086:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55be4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.267095:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55be4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.267102:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760756.267108:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55be4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.267117:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55be4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.267137:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55be4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.267145:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55be4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22294528->22298623] (req 22294528->22298623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.267249:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508379904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 124556us (124834us total) trans 4294972763 rc 0/0 00000100:00100000:1.0:1789760756.267260:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5475 00000100:00100000:1.0:1789760756.267272:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735168094259 time 735247386714 state 0x2 ref 0 00000100:00100000:0.0:1789760756.274721:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760756.274801:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508380032 00000100:00100000:1.0:1789760756.274810:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508380032 00000100:00100000:1.0:1789760756.274861:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735168094259 time 735247386714 state 0x0 ref 1 00000100:00100000:1.0:1789760756.274874:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735247647130 time 735247386714 state 0x1 ref 1 00000100:00100000:1.0:1789760756.274909:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735247647130 time 735379620051 state 0x0 ref 1 00000100:00100000:1.0:1789760756.274920:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508380032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760756.274974:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22298624, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:1.0:1789760756.274982:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55beb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.274996:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760756.275001:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.275007:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55beb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.275020:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55beb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.307508:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972764 is committed 00002000:00010000:1.0:1789760756.307712:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22298624 -> 22302720 00010000:00010000:1.0:1789760756.307727:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55beb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.307739:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55beb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.307754:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55beb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.307761:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760756.307766:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55beb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.307775:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55beb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.307795:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55beb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.307802:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55beb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22298624->22302719] (req 22298624->22302719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.307888:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508380032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32965us (33172us total) trans 4294972764 rc 0/0 00000100:00100000:1.0:1789760756.307898:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5476 00000100:00100000:1.0:1789760756.307908:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735247647130 time 735379620051 state 0x2 ref 0 00000100:00100000:0.0:1789760756.312172:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760756.315441:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508380160 00000100:00100000:1.0:1789760756.315456:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508380160 00000100:00100000:0.0:1789760756.317371:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735247647130 time 735379620051 state 0x0 ref 1 00000100:00100000:0.0:1789760756.317390:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735379880467 time 735379620051 state 0x1 ref 1 00000100:00100000:0.0:1789760756.317438:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735379880467 time 735422148209 state 0x0 ref 1 00000100:00100000:0.0:1789760756.317453:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508380160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760756.317507:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22302720, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:0.0:1789760756.317516:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55bf2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.317533:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760756.317538:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.317545:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.317561:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.343184:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22302720 -> 22306816 00010000:00010000:3.0:1789760756.343206:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.343220:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55bf2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.343237:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55bf2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.343245:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760756.343251:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55bf2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.343261:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55bf2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.343283:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55bf2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.343327:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55bf2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22302720->22306815] (req 22302720->22306815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.343420:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508380160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25965us (31255us total) trans 4294972765 rc 0/0 00000100:00100000:3.0:1789760756.343432:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5477 00000100:00100000:3.0:1789760756.343442:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735379880467 time 735422148209 state 0x2 ref 0 00000001:00080000:1.0:1789760756.343580:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972765 is committed 00000100:00100000:2.0:1789760756.348168:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760756.348277:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508380288 00000100:00100000:1.0:1789760756.348473:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508380288 00000100:00100000:1.0:1789760756.348629:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735379880467 time 735422148209 state 0x0 ref 1 00000100:00100000:1.0:1789760756.348646:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735422408625 time 735422148209 state 0x1 ref 1 00000100:00100000:0.0:1789760756.348763:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735422408625 time 735453471542 state 0x0 ref 1 00000100:00100000:0.0:1789760756.348778:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508380288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760756.348826:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22306816, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:0.0:1789760756.348835:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55bf9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.348850:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760756.348855:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.348862:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55bf9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.348876:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55bf9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760756.409881:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22306816 -> 22310912 00010000:00010000:2.0:1789760756.409907:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55bf9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.409914:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972766 is committed 00010000:00010000:2.0:1789760756.409922:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55bf9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.409931:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55bf9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.409939:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760756.409944:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55bf9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.409954:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55bf9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.409990:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55bf9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.409998:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55bf9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22306816->22310911] (req 22306816->22310911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.410097:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508380288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61317us (61935us total) trans 4294972766 rc 0/0 00000100:00100000:2.0:1789760756.410108:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5478 00000100:00100000:2.0:1789760756.410119:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735422408625 time 735453471542 state 0x2 ref 0 00000100:00100000:2.0:1789760756.416908:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760756.417017:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508380416 00000100:00100000:1.0:1789760756.417027:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508380416 00000100:00100000:1.0:1789760756.417087:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735422408625 time 735453471542 state 0x0 ref 1 00000100:00100000:1.0:1789760756.417104:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735453731958 time 735453471542 state 0x1 ref 1 00000100:00100000:1.0:1789760756.417135:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735453731958 time 735521845543 state 0x0 ref 1 00000100:00100000:1.0:1789760756.417149:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508380416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760756.417201:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22310912, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:1.0:1789760756.417211:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55c00 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.417226:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760756.417231:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.417237:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55c00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.417251:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55c00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.459723:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972767 is committed 00002000:00010000:3.0:1789760756.460049:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22310912 -> 22315008 00010000:00010000:3.0:1789760756.460065:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55c00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.460080:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55c00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.460088:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55c00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.460095:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760756.460099:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55c00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.460109:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55c00 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.460128:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55c00 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.460135:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe55c00 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22310912->22315007] (req 22310912->22315007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.460236:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508380416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43084us (43333us total) trans 4294972767 rc 0/0 00000100:00100000:3.0:1789760756.460245:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5479 00000100:00100000:3.0:1789760756.460257:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735453731958 time 735521845543 state 0x2 ref 0 00000100:00100000:0.0:1789760756.463843:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760756.463970:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508380544 00000100:00100000:3.0:1789760756.463980:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508380544 00000100:00100000:3.0:1789760756.464035:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735453731958 time 735521845543 state 0x0 ref 1 00000100:00100000:3.0:1789760756.464049:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735522105959 time 735521845543 state 0x1 ref 1 00000100:00100000:3.0:1789760756.464078:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735522105959 time 735568788605 state 0x0 ref 1 00000100:00100000:3.0:1789760756.464090:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508380544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760756.464136:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22315008, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:3.0:1789760756.464144:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c07 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.464159:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760756.464163:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.464169:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.464181:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.481072:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760756.496404:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972768 is committed 00002000:00010000:3.0:1789760756.497151:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22315008 -> 22319104 00010000:00010000:3.0:1789760756.497170:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.497183:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.497197:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.497204:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760756.497209:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.497218:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c07 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.497239:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c07 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.497247:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c07 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22315008->22319103] (req 22315008->22319103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.497491:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508380544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33397us (33653us total) trans 4294972768 rc 0/0 00000100:00100000:3.0:1789760756.497501:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5480 00000100:00100000:3.0:1789760756.497511:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735522105959 time 735568788605 state 0x2 ref 0 00000100:00100000:1.0:1789760756.509047:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760756.509136:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508380672 00000100:00100000:2.0:1789760756.509148:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508380672 00000100:00100000:2.0:1789760756.509201:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735522105959 time 735568788605 state 0x0 ref 1 00000100:00100000:2.0:1789760756.509217:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735569049021 time 735568788605 state 0x1 ref 1 00000100:00100000:2.0:1789760756.509249:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735569049021 time 735613959317 state 0x0 ref 1 00000100:00100000:2.0:1789760756.509261:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e67b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508380672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760756.509350:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22319104, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0xb007 00010000:00010000:2.0:1789760756.509360:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.509381:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760756.509385:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.509391:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.509404:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.552828:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972769 is committed 00002000:00010000:2.0:1789760756.553064:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22319104 -> 22323200 00002000:00010000:2.0:1789760756.553069:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760756 -> 1789760757 00002000:00010000:2.0:1789760756.553072:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760756 -> 1789760757 00010000:00010000:2.0:1789760756.553084:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.553098:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.553107:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.553114:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760756.553119:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.553129:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.553150:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c0e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.553158:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22319104->22323199] (req 22319104->22323199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.553253:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e67b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508380672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43989us (44211us total) trans 4294972769 rc 0/0 00000100:00100000:2.0:1789760756.553264:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5481 00000100:00100000:2.0:1789760756.553274:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735569049021 time 735613959317 state 0x2 ref 0 00000100:00100000:2.0:1789760756.561393:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760756.561519:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508380800 00000100:00100000:3.0:1789760756.561528:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508380800 00000100:00100000:3.0:1789760756.561594:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735569049021 time 735613959317 state 0x0 ref 1 00000100:00100000:3.0:1789760756.561609:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735614219733 time 735613959317 state 0x1 ref 1 00000100:00100000:3.0:1789760756.561637:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735614219733 time 735666347413 state 0x0 ref 1 00000100:00100000:3.0:1789760756.561649:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508380800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760756.561696:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22323200, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:3.0:1789760756.561705:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.561721:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760756.561726:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.561733:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.561758:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.601466:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972770 is committed 00002000:00010000:3.0:1789760756.601576:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22323200 -> 22327296 00010000:00010000:3.0:1789760756.601591:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.601605:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.601615:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.601623:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760756.601628:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.601638:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.601658:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.601667:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22323200->22327295] (req 22323200->22327295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.601760:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508380800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40108us (40372us total) trans 4294972770 rc 0/0 00000100:00100000:3.0:1789760756.601770:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5482 00000100:00100000:3.0:1789760756.601779:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735614219733 time 735666347413 state 0x2 ref 0 00000100:00100000:1.0:1789760756.605608:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760756.605754:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508380928 00000100:00100000:3.0:1789760756.605764:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508380928 00000100:00100000:3.0:1789760756.605820:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735614219733 time 735666347413 state 0x0 ref 1 00000100:00100000:3.0:1789760756.605836:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735666607829 time 735666347413 state 0x1 ref 1 00000100:00100000:3.0:1789760756.605868:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735666607829 time 735710577864 state 0x0 ref 1 00000100:00100000:3.0:1789760756.606005:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508380928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760756.606055:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22327296, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:3.0:1789760756.606066:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55c1c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.606081:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760756.606085:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.606091:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55c1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.606106:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55c1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.606997:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048864fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3474:1876699534075136:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607065:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607112:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048864fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534075648:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607142:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607162:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534076160:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607186:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607203:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534076672:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607224:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607247:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:2.0:1789760756.607253:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048864f8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534075264:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607352:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607372:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534075776:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607393:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607410:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534076288:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607430:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607459:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534076800:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760756.607481:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607510:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048864fb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534075392:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607533:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607551:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3472:1876699534075904:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607571:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607586:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3472:1876699534076416:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607605:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607619:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534076928:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760756.607638:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607654:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1789760756.607659:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048864f8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534075520:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607678:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607692:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534076032:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607711:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607727:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534076544:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607747:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760756.607777:0:6459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534075136 00000100:00100000:2.0:1789760756.607784:0:6459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534075136 00000100:00100000:2.0:1789760756.607823:0:6459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 478 00000100:00100000:2.0:1789760756.607831:0:6459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cae850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3474:x1876699534075136:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.607879:0:6459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cae850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3474:x1876699534075136:12345-0@lo:400:kworker.0 Request processed in 48us (813us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.607886:0:6459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 478 00000100:00100000:2.0:1789760756.607917:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534075648 00000100:00100000:2.0:1789760756.607920:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534075648 00000100:00100000:2.0:1789760756.607942:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534075392 00000100:00100000:2.0:1789760756.607946:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534075392 00000100:00100000:2.0:1789760756.607970:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 442 00000100:00100000:2.0:1789760756.607977:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534075648:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.608011:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534075648:12345-0@lo:400:kworker.0 Request processed in 34us (869us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608017:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 442 00000100:00100000:2.0:1789760756.608029:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:2.0:1789760756.608035:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca1e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534075392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.608066:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca1e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534075392:12345-0@lo:400:kworker.0 Request processed in 32us (533us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608072:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:2.0:1789760756.608088:0:8103:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534076160 00000100:00100000:2.0:1789760756.608092:0:8103:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534076160 00000100:00100000:2.0:1789760756.608118:0:8103:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534076672 00000100:00100000:2.0:1789760756.608121:0:8103:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534076672 00000100:00100000:2.0:1789760756.608134:0:8103:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534076032 00000100:00100000:2.0:1789760756.608137:0:8103:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534076032 00000100:00100000:2.0:1789760756.608149:0:8103:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534076544 00000100:00100000:2.0:1789760756.608152:0:8103:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534076544 00000100:00100000:2.0:1789760756.608164:0:8103:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:2.0:1789760756.608170:0:8103:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534076160:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.608202:0:8103:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534076160:12345-0@lo:400:kworker.0 Request processed in 31us (1015us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608209:0:8103:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:2.0:1789760756.608220:0:8103:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 231 00000100:00100000:2.0:1789760756.608226:0:8103:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534076672:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.608253:0:8103:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534076672:12345-0@lo:400:kworker.0 Request processed in 26us (1028us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608259:0:8103:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 231 00000100:00100000:2.0:1789760756.608269:0:8103:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00100000:2.0:1789760756.608275:0:8103:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534076032:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.608324:0:8103:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534076032:12345-0@lo:400:kworker.0 Request processed in 49us (611us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608330:0:8103:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00100000:2.0:1789760756.608340:0:8103:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 233 00000100:00100000:2.0:1789760756.608345:0:8103:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534076544:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.608371:0:8103:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534076544:12345-0@lo:400:kworker.0 Request processed in 25us (622us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608377:0:8103:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 233 00000100:00100000:2.0:1789760756.608397:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534075264 00000100:00100000:2.0:1789760756.608400:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534075264 00000100:00100000:2.0:1789760756.608415:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534075776 00000100:00100000:2.0:1789760756.608418:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534075776 00000100:00100000:2.0:1789760756.608429:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534076288 00000100:00100000:2.0:1789760756.608431:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534076288 00000100:00100000:2.0:1789760756.608441:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534075904 00000100:00100000:2.0:1789760756.608443:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534075904 00000100:00100000:2.0:1789760756.608453:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534076416 00000100:00100000:2.0:1789760756.608455:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534076416 00000100:00100000:2.0:1789760756.608465:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534075520 00000100:00100000:2.0:1789760756.608468:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534075520 00000100:00100000:2.0:1789760756.608481:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 923 00000100:00100000:2.0:1789760756.608486:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534075264:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.608515:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534075264:12345-0@lo:400:kworker.0 Request processed in 29us (1162us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608521:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 923 00000100:00100000:2.0:1789760756.608531:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 924 00000100:00100000:2.0:1789760756.608537:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534075776:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.608559:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534075776:12345-0@lo:400:kworker.0 Request processed in 23us (1166us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608565:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 924 00000100:00100000:2.0:1789760756.608575:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 925 00000100:00100000:2.0:1789760756.608581:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca1ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534076288:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.608604:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca1ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534076288:12345-0@lo:400:kworker.0 Request processed in 23us (1173us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608610:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 925 00000100:00100000:2.0:1789760756.608619:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 926 00000100:00100000:2.0:1789760756.608625:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca1c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3472:x1876699534075904:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.608646:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca1c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3472:x1876699534075904:12345-0@lo:400:kworker.0 Request processed in 22us (1074us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608652:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 926 00000100:00100000:2.0:1789760756.608661:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 927 00000100:00100000:2.0:1789760756.608667:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca1fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3472:x1876699534076416:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.608688:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca1fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3472:x1876699534076416:12345-0@lo:400:kworker.0 Request processed in 22us (1082us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608696:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 927 00000100:00100000:2.0:1789760756.608705:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 928 00000100:00100000:2.0:1789760756.608711:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca1d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534075520:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.608737:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca1d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534075520:12345-0@lo:400:kworker.0 Request processed in 27us (1059us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608742:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 928 00000100:00100000:2.0:1789760756.608844:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534076800 00000100:00100000:2.0:1789760756.608848:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534076800 00000100:00100000:2.0:1789760756.608873:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534076928 00000100:00100000:2.0:1789760756.608876:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534076928 00000100:00100000:2.0:1789760756.608887:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:2.0:1789760756.608892:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534076800:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760756.608906:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3835691008 avail 3802136576 00000100:00100000:2.0:1789760756.608937:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534076800:12345-0@lo:13:osp-pre-0-1.0 Request processed in 44us (1454us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.608942:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00100000:2.0:1789760756.608955:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:2.0:1789760756.608977:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca1ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534076928:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789760756.609000:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760756.609027:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca1ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534076928:12345-0@lo:13:osp-pre-1-0.0 Request processed in 67us (1387us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760756.609033:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00080000:2.0:1789760756.609082:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760756.609086:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760756.609088:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760756.609090:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760756.609137:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048864f8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534075520:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.609151:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534076032:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.609161:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534076544:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.609169:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048864fb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534075392:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.609179:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3472:1876699534075904:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.609188:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3472:1876699534076416:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.609204:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534076928:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760756.609247:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048864f8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534075264:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.609256:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534075776:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.609264:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534076288:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.609274:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534076800:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760756.609282:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048864fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3474:1876699534075136:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.609316:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048864fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534075648:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.609326:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534076160:0@lo:400:kworker.0 00000100:00100000:2.0:1789760756.609334:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534076672:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.611348:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b66f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534077056:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760756.611408:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760756.612809:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534077056 00000100:00100000:0.0:1789760756.612821:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534077056 00000100:00100000:0.0:1789760756.612884:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:0.0:1789760756.612896:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534077056:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760756.612920:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528154624 pending 1703936 free 3835691008 avail 3802136576 00000100:00100000:0.0:1789760756.612996:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534077056:12345-0@lo:13:osp-pre-0-0.0 Request processed in 102us (1586us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760756.613003:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:0.0:1789760756.613056:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b66f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534077056:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760756.613515:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760756.613527:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887e65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534077184:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760756.613566:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760756.613598:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534077184 00000100:00100000:1.0:1789760756.613602:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534077184 00000100:00100000:1.0:1789760756.613625:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000100:00100000:1.0:1789760756.613631:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e67480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534077184:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760756.613643:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760756.613675:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e67480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534077184:12345-0@lo:13:osp-pre-1-1.0 Request processed in 43us (109us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760756.613682:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00000100:00100000:1.0:1789760756.613710:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887e65c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534077184:0@lo:13:osp-pre-1-1.0 00000001:00080000:1.0:1789760756.638898:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972771 is committed 00002000:00010000:2.0:1789760756.639001:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22327296 -> 22331392 00010000:00010000:2.0:1789760756.639016:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55c1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.639033:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55c1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.639042:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55c1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.639050:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760756.639055:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55c1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.639064:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55c1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.639083:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55c1c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.639091:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55c1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22327296->22331391] (req 22327296->22331391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.639187:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508380928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33303us (33584us total) trans 4294972771 rc 0/0 00000100:00100000:2.0:1789760756.639198:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5483 00000100:00100000:2.0:1789760756.639208:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735666607829 time 735710577864 state 0x2 ref 0 00000100:00100000:1.0:1789760756.643335:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760756.643721:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508381056 00000100:00100000:2.0:1789760756.643732:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508381056 00000100:00100000:2.0:1789760756.643786:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735666607829 time 735710577864 state 0x0 ref 1 00000100:00100000:2.0:1789760756.643802:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735710838280 time 735710577864 state 0x1 ref 1 00000100:00100000:2.0:1789760756.643834:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735710838280 time 735748544838 state 0x0 ref 1 00000100:00100000:2.0:1789760756.643846:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508381056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760756.643902:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22331392, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:2.0:1789760756.643911:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c23 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.643925:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760756.643929:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.643935:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.643949:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.676474:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22331392 -> 22335488 00010000:00010000:3.0:1789760756.676492:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.676505:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.676514:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.676522:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760756.676527:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.676542:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c23 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.676561:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c23 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.676569:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55c23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22331392->22335487] (req 22331392->22335487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.676663:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508381056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32813us (33332us total) trans 4294972772 rc 0/0 00000100:00100000:3.0:1789760756.676674:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5484 00000100:00100000:3.0:1789760756.676684:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735710838280 time 735748544838 state 0x2 ref 0 00000001:00080000:1.0:1789760756.676808:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972772 is committed 00000100:00100000:1.0:1789760756.679809:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760756.679894:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508381184 00000100:00100000:3.0:1789760756.679903:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508381184 00000100:00100000:3.0:1789760756.679954:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735710838280 time 735748544838 state 0x0 ref 1 00000100:00100000:3.0:1789760756.679980:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735748805254 time 735748544838 state 0x1 ref 1 00000100:00100000:3.0:1789760756.680008:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735748805254 time 735784719025 state 0x0 ref 1 00000100:00100000:3.0:1789760756.680021:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508381184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760756.680070:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22335488, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:3.0:1789760756.680079:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.680094:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760756.680098:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.680105:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.680120:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760756.704752:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972773 is committed 00002000:00010000:3.0:1789760756.704859:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22335488 -> 22339584 00010000:00010000:3.0:1789760756.704885:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.704898:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.704906:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.704913:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760756.704918:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.704927:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.704946:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c2a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.704955:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22335488->22339583] (req 22335488->22339583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.705056:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508381184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25033us (25252us total) trans 4294972773 rc 0/0 00000100:00100000:3.0:1789760756.705067:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5485 00000100:00100000:3.0:1789760756.705077:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735748805254 time 735784719025 state 0x2 ref 0 00000100:00100000:3.0:1789760756.708536:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760756.708631:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508381312 00000100:00100000:0.0:1789760756.708642:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508381312 00000100:00100000:0.0:1789760756.708693:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735748805254 time 735784719025 state 0x0 ref 1 00000100:00100000:0.0:1789760756.708708:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735784979441 time 735784719025 state 0x1 ref 1 00000100:00100000:0.0:1789760756.708739:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735784979441 time 735813449477 state 0x0 ref 1 00000100:00100000:0.0:1789760756.708751:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508381312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760756.708798:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22339584, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:0.0:1789760756.708806:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c31 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.708822:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760756.708826:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.708833:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.708847:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760756.738244:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22339584 -> 22343680 00010000:00010000:0.0:1789760756.738263:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.738276:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.738305:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.738324:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760756.738329:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.738339:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.738359:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c31 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.738368:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22339584->22343679] (req 22339584->22343679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.738460:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508381312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29706us (29929us total) trans 4294972774 rc 0/0 00000100:00100000:0.0:1789760756.738470:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5486 00000100:00100000:0.0:1789760756.738480:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735784979441 time 735813449477 state 0x2 ref 0 00000001:00080000:3.0:1789760756.738559:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972774 is committed 00000100:00100000:0.0:1789760756.742069:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760756.742173:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508381440 00000100:00100000:3.0:1789760756.742186:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508381440 00000100:00100000:3.0:1789760756.742241:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735784979441 time 735813449477 state 0x0 ref 1 00000100:00100000:3.0:1789760756.742256:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735813709893 time 735813449477 state 0x1 ref 1 00000100:00100000:3.0:1789760756.742408:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735813709893 time 735846995439 state 0x0 ref 1 00000100:00100000:3.0:1789760756.742424:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508381440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760756.742471:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22343680, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:3.0:1789760756.742479:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.742495:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760756.742499:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.742507:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.742520:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.775271:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972775 is committed 00002000:00010000:0.0:1789760756.775472:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22343680 -> 22347776 00010000:00010000:0.0:1789760756.775488:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.775500:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.775510:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.775519:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760756.775524:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.775533:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.775553:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c38 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.775625:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55c38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22343680->22347775] (req 22343680->22347775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.775753:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508381440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33328us (33689us total) trans 4294972775 rc 0/0 00000100:00100000:0.0:1789760756.775768:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5487 00000100:00100000:0.0:1789760756.775778:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735813709893 time 735846995439 state 0x2 ref 0 00000100:00100000:3.0:1789760756.780781:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760756.780848:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508381568 00000100:00100000:0.0:1789760756.780857:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508381568 00000100:00100000:0.0:1789760756.780924:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735813709893 time 735846995439 state 0x0 ref 1 00000100:00100000:0.0:1789760756.780940:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735847255855 time 735846995439 state 0x1 ref 1 00000100:00100000:0.0:1789760756.780980:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735847255855 time 735885690724 state 0x0 ref 1 00000100:00100000:0.0:1789760756.780992:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508381568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760756.781041:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22347776, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:0.0:1789760756.781050:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c3f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.781065:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760756.781069:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.781076:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.781091:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.810811:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972776 is committed 00002000:00010000:0.0:1789760756.810914:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22347776 -> 22351872 00010000:00010000:0.0:1789760756.810929:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.810942:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.810951:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.810967:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760756.810972:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.810980:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.810999:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c3f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.811007:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55c3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22347776->22351871] (req 22347776->22351871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.811099:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508381568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30103us (30324us total) trans 4294972776 rc 0/0 00000100:00100000:0.0:1789760756.811109:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5488 00000100:00100000:0.0:1789760756.811119:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735847255855 time 735885690724 state 0x2 ref 0 00000100:00100000:1.0:1789760756.814676:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760756.814790:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508381696 00000100:00100000:0.0:1789760756.814799:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508381696 00000100:00100000:0.0:1789760756.814851:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735847255855 time 735885690724 state 0x0 ref 1 00000100:00100000:0.0:1789760756.814867:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735885951140 time 735885690724 state 0x1 ref 1 00000100:00100000:0.0:1789760756.815001:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735885951140 time 735919709928 state 0x0 ref 1 00000100:00100000:0.0:1789760756.815015:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508381696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760756.815067:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22351872, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:0.0:1789760756.815076:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55c46 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.815091:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760756.815095:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.815103:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55c46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.815118:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55c46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760756.845829:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22351872 -> 22355968 00010000:00010000:1.0:1789760756.845850:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55c46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.845864:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55c46 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.845955:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55c46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.845971:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760756.845976:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55c46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.845986:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55c46 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.846006:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55c46 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.846014:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55c46 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22351872->22355967] (req 22351872->22355967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760756.846106:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508381696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31088us (31435us total) trans 4294972777 rc 0/0 00000100:00100000:1.0:1789760756.846116:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5489 00000100:00100000:1.0:1789760756.846126:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735885951140 time 735919709928 state 0x2 ref 0 00000001:00080000:3.0:1789760756.848430:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972777 is committed 00000100:00100000:1.0:1789760756.849919:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760756.850058:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508381824 00000100:00100000:0.0:1789760756.850069:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508381824 00000100:00100000:0.0:1789760756.850126:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735885951140 time 735919709928 state 0x0 ref 1 00000100:00100000:0.0:1789760756.850143:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735919970344 time 735919709928 state 0x1 ref 1 00000100:00100000:0.0:1789760756.850178:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735919970344 time 735954889111 state 0x0 ref 1 00000100:00100000:0.0:1789760756.850194:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508381824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760756.850243:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22355968, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:0.0:1789760756.850254:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c4d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.850271:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760756.850276:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.850283:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.850431:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760756.883814:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972778 is committed 00002000:00010000:0.0:1789760756.884022:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22355968 -> 22360064 00010000:00010000:0.0:1789760756.884037:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.884049:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.884057:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.884064:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760756.884068:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.884077:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.884095:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c4d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.884102:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22355968->22360063] (req 22355968->22360063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.884188:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508381824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33990us (34272us total) trans 4294972778 rc 0/0 00000100:00100000:0.0:1789760756.884197:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5490 00000100:00100000:0.0:1789760756.884207:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735919970344 time 735954889111 state 0x2 ref 0 00000100:00100000:3.0:1789760756.887261:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760756.887364:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508381952 00000100:00100000:1.0:1789760756.887373:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508381952 00000100:00100000:1.0:1789760756.887431:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735919970344 time 735954889111 state 0x0 ref 1 00000100:00100000:1.0:1789760756.887450:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735955149527 time 735954889111 state 0x1 ref 1 00000100:00100000:1.0:1789760756.887481:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735955149527 time 735992191529 state 0x0 ref 1 00000100:00100000:1.0:1789760756.887492:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508381952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760756.887539:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22360064, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:1.0:1789760756.887549:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55c54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.887564:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760756.887569:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760756.887575:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55c54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760756.887588:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55c54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760756.920552:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22360064 -> 22364160 00010000:00010000:0.0:1789760756.920572:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55c54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.920585:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55c54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.920594:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55c54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.920601:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760756.920607:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55c54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.920616:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55c54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.920637:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972779 is committed 00010000:00010000:0.0:1789760756.920638:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55c54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.920645:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55c54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22360064->22364159] (req 22360064->22364159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760756.920752:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508381952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33249us (33488us total) trans 4294972779 rc 0/0 00000100:00100000:0.0:1789760756.920763:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5491 00000100:00100000:0.0:1789760756.920774:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735955149527 time 735992191529 state 0x2 ref 0 00000100:00100000:0.0:1789760756.926160:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760756.926654:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508382080 00000100:00100000:1.0:1789760756.926663:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508382080 00000100:00100000:2.0:1789760756.926814:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735955149527 time 735992191529 state 0x0 ref 1 00000100:00100000:2.0:1789760756.926829:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735992451945 time 735992191529 state 0x1 ref 1 00000100:00100000:2.0:1789760756.926890:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735992451945 time 736031600130 state 0x0 ref 1 00000100:00100000:2.0:1789760756.926905:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508382080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760756.926948:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22364160, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:2.0:1789760756.926965:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.926980:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760756.926985:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.926990:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.927003:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.954704:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972780 is committed 00002000:00010000:2.0:1789760756.954866:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22364160 -> 22368256 00010000:00010000:2.0:1789760756.954881:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.954894:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.954904:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.954912:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760756.954917:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.954927:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c5b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.954946:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c5b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.954954:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22364160->22368255] (req 22364160->22368255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760756.955139:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508382080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28230us (28984us total) trans 4294972780 rc 0/0 00000100:00100000:2.0:1789760756.955150:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5492 00000100:00100000:2.0:1789760756.955160:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735992451945 time 736031600130 state 0x2 ref 0 00000100:00100000:1.0:1789760756.959817:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760756.959904:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508382208 00000100:00100000:0.0:1789760756.959913:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508382208 00000100:00100000:0.0:1789760756.959976:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 735992451945 time 736031600130 state 0x0 ref 1 00000100:00100000:0.0:1789760756.959991:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736031860546 time 736031600130 state 0x1 ref 1 00000100:00100000:0.0:1789760756.960047:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736031860546 time 736064756892 state 0x0 ref 1 00000100:00100000:0.0:1789760756.960060:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508382208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760756.960109:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22368256, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:0.0:1789760756.960118:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.960133:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760756.960137:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760756.960144:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760756.960159:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760756.991246:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972781 is committed 00002000:00010000:3.0:1789760756.991505:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22368256 -> 22372352 00010000:00010000:3.0:1789760756.991520:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.991534:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.991544:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.991553:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760756.991558:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.991568:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.991592:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c62 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.991599:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55c62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22368256->22372351] (req 22368256->22372351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.991694:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508382208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31631us (31882us total) trans 4294972781 rc 0/0 00000100:00100000:3.0:1789760756.991704:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5493 00000100:00100000:3.0:1789760756.991715:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736031860546 time 736064756892 state 0x2 ref 0 00000100:00100000:2.0:1789760756.995999:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760756.996124:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508382336 00000100:00100000:0.0:1789760756.996133:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508382336 00000100:00100000:0.0:1789760756.996186:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736031860546 time 736064756892 state 0x0 ref 1 00000100:00100000:0.0:1789760756.996202:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736065017308 time 736064756892 state 0x1 ref 1 00000100:00100000:0.0:1789760756.996233:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736065017308 time 736100943757 state 0x0 ref 1 00000100:00100000:0.0:1789760756.996245:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508382336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760756.996586:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22372352, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:2.0:1789760756.996597:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55c69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.996611:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760756.996615:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760756.996621:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55c69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760756.996635:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55c69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760757.032779:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972782 is committed 00002000:00010000:3.0:1789760757.032944:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22372352 -> 22376448 00010000:00010000:3.0:1789760757.032968:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55c69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.032980:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55c69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.032989:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55c69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.032995:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760757.032999:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55c69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.033011:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55c69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.033033:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55c69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.033039:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55c69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22372352->22376447] (req 22372352->22376447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.033168:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508382336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36919us (37174us total) trans 4294972782 rc 0/0 00000100:00100000:3.0:1789760757.033178:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5494 00000100:00100000:3.0:1789760757.033188:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736065017308 time 736100943757 state 0x2 ref 0 00000100:00100000:0.0:1789760757.036988:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760757.037078:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508382464 00000100:00100000:2.0:1789760757.037088:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508382464 00000100:00100000:2.0:1789760757.037158:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736065017308 time 736100943757 state 0x0 ref 1 00000100:00100000:2.0:1789760757.037173:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736101204173 time 736100943757 state 0x1 ref 1 00000100:00100000:2.0:1789760757.037204:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736101204173 time 736141914759 state 0x0 ref 1 00000100:00100000:2.0:1789760757.037217:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508382464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760757.037263:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22376448, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:2.0:1789760757.037272:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.037326:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760757.037331:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.037337:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.037352:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760757.040024:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760757.067913:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972783 is committed 00002000:00010000:2.0:1789760757.068138:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22376448 -> 22380544 00010000:00010000:2.0:1789760757.068155:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.068173:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.068183:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.068191:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760757.068196:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.068205:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c70 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.068226:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c70 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.068234:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22376448->22380543] (req 22376448->22380543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.068371:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508382464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31151us (31388us total) trans 4294972783 rc 0/0 00000100:00100000:2.0:1789760757.068382:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5495 00000100:00100000:2.0:1789760757.068393:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736101204173 time 736141914759 state 0x2 ref 0 00000100:00100000:0.0:1789760757.075998:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760757.076461:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508382592 00000100:00100000:3.0:1789760757.076472:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508382592 00000100:00100000:3.0:1789760757.076530:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736101204173 time 736141914759 state 0x0 ref 1 00000100:00100000:3.0:1789760757.076546:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736142175175 time 736141914759 state 0x1 ref 1 00000100:00100000:2.0:1789760757.076841:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736142175175 time 736181548600 state 0x0 ref 1 00000100:00100000:2.0:1789760757.076860:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508382592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760757.076912:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22380544, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:2.0:1789760757.076920:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.076935:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760757.076940:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.076947:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.076973:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.118660:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972784 is committed 00002000:00010000:0.0:1789760757.118770:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22380544 -> 22384640 00010000:00010000:0.0:1789760757.118785:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.118798:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.118807:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.118814:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760757.118819:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.118828:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.118848:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.118855:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe55c77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22380544->22384639] (req 22380544->22384639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.118944:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508382592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42084us (42954us total) trans 4294972784 rc 0/0 00000100:00100000:0.0:1789760757.118953:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5496 00000100:00100000:0.0:1789760757.118972:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736142175175 time 736181548600 state 0x2 ref 0 00000100:00100000:1.0:1789760757.125472:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760757.125604:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508382720 00000100:00100000:3.0:1789760757.125614:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508382720 00000100:00100000:3.0:1789760757.126500:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736142175175 time 736181548600 state 0x0 ref 1 00000100:00100000:3.0:1789760757.126519:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736181809016 time 736181548600 state 0x1 ref 1 00000100:00100000:3.0:1789760757.126610:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736181809016 time 736231319859 state 0x0 ref 1 00000100:00100000:3.0:1789760757.126629:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508382720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760757.126690:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22384640, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:3.0:1789760757.126699:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55c7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.126715:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760757.126720:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.126726:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55c7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.126740:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55c7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760757.155717:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22384640 -> 22388736 00010000:00010000:0.0:1789760757.155738:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55c7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.155752:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55c7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.155761:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55c7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.155768:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760757.155773:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55c7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.155782:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55c7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.155802:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55c7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.155809:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55c7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22384640->22388735] (req 22384640->22388735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.155837:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972785 is committed 00000100:00100000:0.0:1789760757.155893:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508382720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29267us (30425us total) trans 4294972785 rc 0/0 00000100:00100000:0.0:1789760757.155903:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5497 00000100:00100000:0.0:1789760757.155912:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736181809016 time 736231319859 state 0x2 ref 0 00000100:00100000:1.0:1789760757.161420:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760757.161766:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508382848 00000100:00100000:2.0:1789760757.161776:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508382848 00000100:00100000:2.0:1789760757.161834:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736181809016 time 736231319859 state 0x0 ref 1 00000100:00100000:2.0:1789760757.161852:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736231580275 time 736231319859 state 0x1 ref 1 00000100:00100000:2.0:1789760757.162355:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736231580275 time 736267061645 state 0x0 ref 1 00000100:00100000:2.0:1789760757.162375:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508382848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760757.162433:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22388736, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:2.0:1789760757.162443:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55c85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.162464:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760757.162469:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.162476:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55c85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.162492:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55c85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.200406:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972786 is committed 00002000:00010000:0.0:1789760757.200530:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22388736 -> 22392832 00010000:00010000:0.0:1789760757.200545:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55c85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.200558:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55c85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.200568:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55c85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.200575:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760757.200580:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55c85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.200589:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55c85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.200608:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55c85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.200615:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55c85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22388736->22392831] (req 22388736->22392831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.200706:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508382848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38334us (39290us total) trans 4294972786 rc 0/0 00000100:00100000:0.0:1789760757.200716:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5498 00000100:00100000:0.0:1789760757.200727:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736231580275 time 736267061645 state 0x2 ref 0 00000100:00100000:1.0:1789760757.205916:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760757.206024:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508382976 00000100:00100000:0.0:1789760757.206035:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508382976 00000100:00100000:0.0:1789760757.206088:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736231580275 time 736267061645 state 0x0 ref 1 00000100:00100000:0.0:1789760757.206104:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736267322061 time 736267061645 state 0x1 ref 1 00000100:00100000:0.0:1789760757.206129:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736267322061 time 736310839453 state 0x0 ref 1 00000100:00100000:0.0:1789760757.206141:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508382976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760757.206199:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22392832, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:0.0:1789760757.206208:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.206223:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.206228:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.206234:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.206247:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760757.244802:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22392832 -> 22396928 00010000:00010000:0.0:1789760757.244825:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.244845:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.244855:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.244862:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760757.244866:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.244876:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c8c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.244895:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c8c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.244902:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c8c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22392832->22396927] (req 22392832->22396927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.245012:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508382976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38867us (39099us total) trans 4294972787 rc 0/0 00000100:00100000:0.0:1789760757.245024:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5499 00000100:00100000:0.0:1789760757.245034:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736267322061 time 736310839453 state 0x2 ref 0 00000001:00080000:3.0:1789760757.245264:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972787 is committed 00000100:00100000:1.0:1789760757.248432:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760757.248497:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508383104 00000100:00100000:0.0:1789760757.248507:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508383104 00000100:00100000:0.0:1789760757.248556:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736267322061 time 736310839453 state 0x0 ref 1 00000100:00100000:0.0:1789760757.248571:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736311099869 time 736310839453 state 0x1 ref 1 00000100:00100000:0.0:1789760757.248599:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736311099869 time 736353310529 state 0x0 ref 1 00000100:00100000:0.0:1789760757.248612:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d41850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508383104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760757.248662:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22396928, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:0.0:1789760757.248671:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55c93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.248687:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.248692:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.248698:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55c93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.248713:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55c93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.275780:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22396928 -> 22401024 00010000:00010000:3.0:1789760757.275799:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55c93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.275812:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55c93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.275822:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55c93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.275829:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760757.275833:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55c93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.275843:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55c93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.275980:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972788 is committed 00010000:00010000:3.0:1789760757.276020:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55c93 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.276031:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55c93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22396928->22401023] (req 22396928->22401023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.276154:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d41850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508383104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27539us (27727us total) trans 4294972788 rc 0/0 00000100:00100000:3.0:1789760757.276165:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5500 00000100:00100000:3.0:1789760757.276174:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736311099869 time 736353310529 state 0x2 ref 0 00000100:00100000:1.0:1789760757.279233:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760757.279531:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508383232 00000100:00100000:0.0:1789760757.279541:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508383232 00000100:00100000:0.0:1789760757.279596:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736311099869 time 736353310529 state 0x0 ref 1 00000100:00100000:0.0:1789760757.279612:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736353570945 time 736353310529 state 0x1 ref 1 00000100:00100000:0.0:1789760757.279690:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736353570945 time 736384400304 state 0x0 ref 1 00000100:00100000:0.0:1789760757.279706:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508383232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760757.279750:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22401024, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:0.0:1789760757.279759:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c9a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.279775:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.279779:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.279785:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.279799:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.312123:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972789 is committed 00002000:00010000:1.0:1789760757.312214:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22401024 -> 22405120 00010000:00010000:1.0:1789760757.312229:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.312240:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.312248:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.312254:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760757.312259:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.312267:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.312671:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.312683:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe55c9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22401024->22405119] (req 22401024->22405119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.312931:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508383232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33157us (33637us total) trans 4294972789 rc 0/0 00000100:00100000:1.0:1789760757.312944:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5501 00000100:00100000:1.0:1789760757.312963:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736353570945 time 736384400304 state 0x2 ref 0 00000100:00100000:1.0:1789760757.319598:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760757.319696:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508383360 00000100:00100000:0.0:1789760757.319706:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508383360 00000100:00100000:0.0:1789760757.319762:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736353570945 time 736384400304 state 0x0 ref 1 00000100:00100000:0.0:1789760757.319779:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736384660720 time 736384400304 state 0x1 ref 1 00000100:00100000:0.0:1789760757.319807:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736384660720 time 736424518088 state 0x0 ref 1 00000100:00100000:0.0:1789760757.319820:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508383360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760757.319871:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22405120, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:0.0:1789760757.319880:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55ca1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.319902:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.319907:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.319914:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55ca1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.319927:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55ca1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760757.380719:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22405120 -> 22409216 00010000:00010000:0.0:1789760757.380740:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55ca1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.380753:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55ca1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.380763:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55ca1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.380770:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760757.380784:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55ca1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.380795:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55ca1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.380814:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55ca1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.380823:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55ca1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22405120->22409215] (req 22405120->22409215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.380914:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508383360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61092us (61322us total) trans 4294972790 rc 0/0 00000100:00100000:0.0:1789760757.380925:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5502 00000100:00100000:0.0:1789760757.380935:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736384660720 time 736424518088 state 0x2 ref 0 00000001:00080000:3.0:1789760757.382794:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972790 is committed 00000100:00100000:2.0:1789760757.388036:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760757.388415:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508383488 00000100:00100000:3.0:1789760757.388424:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508383488 00000100:00100000:3.0:1789760757.388488:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736384660720 time 736424518088 state 0x0 ref 1 00000100:00100000:3.0:1789760757.388504:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736424778504 time 736424518088 state 0x1 ref 1 00000100:00100000:3.0:1789760757.388534:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736424778504 time 736493244367 state 0x0 ref 1 00000100:00100000:3.0:1789760757.388548:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508383488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760757.388598:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22409216, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:3.0:1789760757.388607:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ca8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.388621:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760757.388626:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.388633:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ca8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.388648:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ca8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760757.436938:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22409216 -> 22413312 00010000:00010000:0.0:1789760757.436969:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ca8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.436983:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ca8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.436991:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ca8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.436998:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760757.437002:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ca8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.437010:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ca8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.437032:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ca8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.437039:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ca8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22409216->22413311] (req 22409216->22413311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.437136:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508383488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48586us (49106us total) trans 4294972791 rc 0/0 00000100:00100000:0.0:1789760757.437146:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5503 00000100:00100000:0.0:1789760757.437155:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736424778504 time 736493244367 state 0x2 ref 0 00000001:00080000:3.0:1789760757.437281:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972791 is committed 00000100:00100000:1.0:1789760757.444537:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760757.444695:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508383616 00000100:00100000:0.0:1789760757.444705:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508383616 00000100:00100000:0.0:1789760757.444759:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736424778504 time 736493244367 state 0x0 ref 1 00000100:00100000:0.0:1789760757.444776:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736493504783 time 736493244367 state 0x1 ref 1 00000100:00100000:0.0:1789760757.444849:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736493504783 time 736549558393 state 0x0 ref 1 00000100:00100000:0.0:1789760757.444872:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508383616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760757.444921:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22413312, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:0.0:1789760757.444932:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55caf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.444949:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.444964:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.444972:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55caf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.444986:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55caf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.506751:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760757.507191:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972792 is committed 00002000:00010000:0.0:1789760757.507518:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22413312 -> 22417408 00010000:00010000:0.0:1789760757.507533:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55caf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.507544:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55caf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.507553:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55caf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.507561:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760757.507565:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55caf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.507574:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55caf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.507592:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55caf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.507605:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55caf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22413312->22417407] (req 22413312->22417407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.507684:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508383616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 62819us (63151us total) trans 4294972792 rc 0/0 00000100:00100000:0.0:1789760757.507694:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5504 00000100:00100000:0.0:1789760757.507703:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736493504783 time 736549558393 state 0x2 ref 0 00000100:00100000:2.0:1789760757.511770:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760757.511854:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508383744 00000100:00100000:0.0:1789760757.511864:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508383744 00000100:00100000:0.0:1789760757.511919:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736493504783 time 736549558393 state 0x0 ref 1 00000100:00100000:0.0:1789760757.511934:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736549818809 time 736549558393 state 0x1 ref 1 00000100:00100000:0.0:1789760757.511972:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736549818809 time 736616682582 state 0x0 ref 1 00000100:00100000:0.0:1789760757.511984:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508383744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760757.512030:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22417408, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0xb007 00010000:00010000:0.0:1789760757.512039:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55cb6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.512055:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.512060:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.512066:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55cb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.512085:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55cb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760757.578172:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22417408 -> 22421504 00002000:00010000:1.0:1789760757.578181:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760757 -> 1789760758 00002000:00010000:1.0:1789760757.578183:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760757 -> 1789760758 00010000:00010000:1.0:1789760757.578196:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55cb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.578208:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55cb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.578217:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55cb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.578224:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760757.578229:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55cb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.578239:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55cb6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.578258:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55cb6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.578271:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55cb6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22417408->22421503] (req 22417408->22421503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.578586:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508383744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 66599us (66825us total) trans 4294972793 rc 0/0 00000100:00100000:1.0:1789760757.578598:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5505 00000100:00100000:1.0:1789760757.578607:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736549818809 time 736616682582 state 0x2 ref 0 00000001:00080000:2.0:1789760757.579029:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972793 is committed 00000100:00100000:2.0:1789760757.583597:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760757.583678:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508383872 00000100:00100000:3.0:1789760757.583688:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508383872 00000100:00100000:3.0:1789760757.583738:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736549818809 time 736616682582 state 0x0 ref 1 00000100:00100000:3.0:1789760757.583753:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736616942998 time 736616682582 state 0x1 ref 1 00000100:00100000:3.0:1789760757.583778:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736616942998 time 736688488818 state 0x0 ref 1 00000100:00100000:3.0:1789760757.583790:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508383872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760757.583841:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22421504, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:3.0:1789760757.583851:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55cbd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.583881:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760757.583885:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.583892:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55cbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.583906:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55cbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760757.615227:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972794 is committed 00002000:00010000:1.0:1789760757.615669:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22421504 -> 22425600 00010000:00010000:1.0:1789760757.615685:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55cbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.615699:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55cbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.615707:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55cbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.615714:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760757.615720:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55cbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.615728:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55cbd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.615751:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55cbd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.615758:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55cbd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22421504->22425599] (req 22421504->22425599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.615968:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508383872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32167us (32366us total) trans 4294972794 rc 0/0 00000100:00100000:1.0:1789760757.615979:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5506 00000100:00100000:1.0:1789760757.615989:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736616942998 time 736688488818 state 0x2 ref 0 00000100:00100000:3.0:1789760757.619718:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760757.619994:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508384000 00000100:00100000:0.0:1789760757.620004:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508384000 00000100:00100000:0.0:1789760757.620059:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736616942998 time 736688488818 state 0x0 ref 1 00000100:00100000:0.0:1789760757.620074:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736688749234 time 736688488818 state 0x1 ref 1 00000100:00100000:0.0:1789760757.620108:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736688749234 time 736724818757 state 0x0 ref 1 00000100:00100000:0.0:1789760757.620120:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508384000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760757.620174:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22425600, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:0.0:1789760757.620183:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55cc4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.620199:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760757.620203:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760757.620210:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55cc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760757.620224:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55cc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760757.632082:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499ca5f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534077312:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760757.632161:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760757.632310:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534077312 00000100:00100000:0.0:1789760757.632317:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534077312 00000100:00100000:0.0:1789760757.632365:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:0.0:1789760757.632374:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534077312:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760757.632804:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789760757.632964:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534077312:12345-0@lo:41:osp-pre-0-1.0 Request processed in 590us (802us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760757.632973:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:1.0:1789760757.633010:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499ca5f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534077312:0@lo:41:osp-pre-0-1.0 00000001:00080000:0.0:1789760757.663180:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972795 is committed 00002000:00010000:1.0:1789760757.663345:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22425600 -> 22429696 00010000:00010000:1.0:1789760757.663359:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55cc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.663372:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55cc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.663381:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55cc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.663388:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760757.663393:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55cc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.663407:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55cc4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.663426:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55cc4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.663434:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55cc4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22425600->22429695] (req 22425600->22429695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.663527:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508384000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43405us (43814us total) trans 4294972795 rc 0/0 00000100:00100000:1.0:1789760757.663538:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5507 00000100:00100000:1.0:1789760757.663547:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736688749234 time 736724818757 state 0x2 ref 0 00000100:00100000:2.0:1789760757.671154:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760757.671242:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508384128 00000100:00100000:1.0:1789760757.671257:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508384128 00000100:00100000:1.0:1789760757.671323:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736688749234 time 736724818757 state 0x0 ref 1 00000100:00100000:1.0:1789760757.671335:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736725079173 time 736724818757 state 0x1 ref 1 00000100:00100000:1.0:1789760757.671356:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736725079173 time 736776068602 state 0x0 ref 1 00000100:00100000:1.0:1789760757.671364:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508384128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760757.671407:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22429696, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:1.0:1789760757.671417:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55ccb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.671433:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760757.671437:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.671443:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55ccb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.671456:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55ccb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760757.714948:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22429696 -> 22433792 00010000:00010000:1.0:1789760757.714977:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55ccb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.714990:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55ccb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.714999:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55ccb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.715007:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760757.715011:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55ccb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.715020:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55ccb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.715040:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55ccb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.715050:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55ccb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22429696->22433791] (req 22429696->22433791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.715149:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508384128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43780us (43997us total) trans 4294972796 rc 0/0 00000100:00100000:1.0:1789760757.715159:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5508 00000100:00100000:1.0:1789760757.715169:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736725079173 time 736776068602 state 0x2 ref 0 00000001:00080000:3.0:1789760757.715861:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972796 is committed 00000100:00100000:2.0:1789760757.719885:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760757.720028:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508384256 00000100:00100000:1.0:1789760757.720037:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508384256 00000100:00100000:1.0:1789760757.720089:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736725079173 time 736776068602 state 0x0 ref 1 00000100:00100000:1.0:1789760757.720104:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736776329018 time 736776068602 state 0x1 ref 1 00000100:00100000:1.0:1789760757.720167:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736776329018 time 736824876988 state 0x0 ref 1 00000100:00100000:1.0:1789760757.720179:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508384256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760757.720228:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22433792, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:1.0:1789760757.720237:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55cd2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.720253:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760757.720262:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.720268:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55cd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.720337:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55cd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760757.751389:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972797 is committed 00002000:00010000:3.0:1789760757.751791:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22433792 -> 22437888 00010000:00010000:3.0:1789760757.751807:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55cd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.751820:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55cd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.751829:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55cd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.751836:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760757.751842:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55cd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.751850:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55cd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.751874:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55cd2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.751882:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55cd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22433792->22437887] (req 22433792->22437887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.752043:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508384256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31860us (32161us total) trans 4294972797 rc 0/0 00000100:00100000:3.0:1789760757.752053:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5509 00000100:00100000:3.0:1789760757.752063:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736776329018 time 736824876988 state 0x2 ref 0 00000100:00100000:0.0:1789760757.756888:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760757.756919:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508384384 00000100:00100000:1.0:1789760757.756929:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508384384 00000100:00100000:1.0:1789760757.756999:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736776329018 time 736824876988 state 0x0 ref 1 00000100:00100000:1.0:1789760757.757015:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736825137404 time 736824876988 state 0x1 ref 1 00000100:00100000:1.0:1789760757.757044:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736825137404 time 736861754114 state 0x0 ref 1 00000100:00100000:1.0:1789760757.757056:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508384384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760757.757106:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22437888, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:1.0:1789760757.757115:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55cd9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.757131:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760757.757135:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.757142:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55cd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.757159:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55cd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.794238:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972798 is committed 00002000:00010000:2.0:1789760757.794351:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22437888 -> 22441984 00010000:00010000:2.0:1789760757.794366:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55cd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.794378:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55cd9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.794388:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55cd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.794400:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760757.794405:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55cd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.794415:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55cd9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.794434:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55cd9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.794441:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55cd9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22437888->22441983] (req 22437888->22441983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.794537:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508384384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37479us (37654us total) trans 4294972798 rc 0/0 00000100:00100000:2.0:1789760757.794548:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5510 00000100:00100000:2.0:1789760757.794557:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736825137404 time 736861754114 state 0x2 ref 0 00000100:00100000:2.0:1789760757.797121:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760757.797203:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508384512 00000100:00100000:1.0:1789760757.797213:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508384512 00000100:00100000:1.0:1789760757.797262:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736825137404 time 736861754114 state 0x0 ref 1 00000100:00100000:1.0:1789760757.797276:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736862014530 time 736861754114 state 0x1 ref 1 00000100:00100000:1.0:1789760757.797335:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736862014530 time 736902044313 state 0x0 ref 1 00000100:00100000:1.0:1789760757.797348:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508384512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760757.797392:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22441984, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:1.0:1789760757.797400:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55ce0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.797415:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760757.797420:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.797427:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55ce0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.797440:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55ce0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.835779:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972799 is committed 00002000:00010000:2.0:1789760757.835867:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22441984 -> 22446080 00010000:00010000:2.0:1789760757.835882:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55ce0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.835895:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55ce0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.835910:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55ce0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.835918:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760757.835923:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55ce0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.835932:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55ce0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.836035:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55ce0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.836044:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55ce0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22441984->22446079] (req 22441984->22446079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760757.836168:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508384512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38817us (39051us total) trans 4294972799 rc 0/0 00000100:00100000:2.0:1789760757.836179:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5511 00000100:00100000:2.0:1789760757.836189:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736862014530 time 736902044313 state 0x2 ref 0 00000100:00100000:1.0:1789760757.842270:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760757.842522:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508384640 00000100:00100000:3.0:1789760757.842532:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508384640 00000100:00100000:3.0:1789760757.842584:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736862014530 time 736902044313 state 0x0 ref 1 00000100:00100000:3.0:1789760757.842595:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736902304729 time 736902044313 state 0x1 ref 1 00000100:00100000:3.0:1789760757.842662:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736902304729 time 736947371674 state 0x0 ref 1 00000100:00100000:3.0:1789760757.842675:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508384640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760757.842723:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22446080, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:3.0:1789760757.842733:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55ce7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.842763:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760757.842768:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.842774:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55ce7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.842789:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55ce7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.878772:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22446080 -> 22450176 00010000:00010000:3.0:1789760757.878792:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55ce7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.878806:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55ce7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.878816:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55ce7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.878823:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760757.878828:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55ce7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.878838:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55ce7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.878858:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55ce7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.878866:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55ce7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22446080->22450175] (req 22446080->22450175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.878961:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508384640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36276us (36686us total) trans 4294972800 rc 0/0 00000100:00100000:3.0:1789760757.878971:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5512 00000100:00100000:3.0:1789760757.878979:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736902304729 time 736947371674 state 0x2 ref 0 00000001:00080000:2.0:1789760757.879119:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972800 is committed 00000100:00100000:0.0:1789760757.882824:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760757.884265:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508384768 00000100:00100000:2.0:1789760757.884279:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508384768 00000100:00100000:2.0:1789760757.884353:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736902304729 time 736947371674 state 0x0 ref 1 00000100:00100000:2.0:1789760757.884372:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736947632090 time 736947371674 state 0x1 ref 1 00000100:00100000:2.0:1789760757.884408:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736947632090 time 736989118127 state 0x0 ref 1 00000100:00100000:2.0:1789760757.884420:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508384768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760757.884471:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22450176, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:2.0:1789760757.884480:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55cee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.884494:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760757.884499:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.884504:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.884518:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.924216:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972801 is committed 00002000:00010000:3.0:1789760757.924485:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22450176 -> 22454272 00010000:00010000:3.0:1789760757.924500:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.924513:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55cee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.924523:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55cee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.924531:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760757.924536:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55cee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.924545:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55cee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.924573:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55cee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.924582:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55cee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22450176->22454271] (req 22450176->22454271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.924668:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508384768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40245us (41848us total) trans 4294972801 rc 0/0 00000100:00100000:3.0:1789760757.924679:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5513 00000100:00100000:3.0:1789760757.924689:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736947632090 time 736989118127 state 0x2 ref 0 00000100:00100000:1.0:1789760757.940909:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760757.941437:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508384896 00000100:00100000:2.0:1789760757.941447:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508384896 00000100:00100000:2.0:1789760757.941499:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736947632090 time 736989118127 state 0x0 ref 1 00000100:00100000:2.0:1789760757.941513:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736989378543 time 736989118127 state 0x1 ref 1 00000100:00100000:2.0:1789760757.941542:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736989378543 time 737046253451 state 0x0 ref 1 00000100:00100000:2.0:1789760757.941554:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508384896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760757.941599:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22454272, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:2.0:1789760757.941607:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55cf5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.941623:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760757.941627:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760757.941633:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55cf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760757.941647:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55cf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760757.975674:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22454272 -> 22458368 00010000:00010000:1.0:1789760757.975694:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55cf5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.975708:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55cf5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.975717:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55cf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.975723:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760757.975727:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55cf5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.975736:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55cf5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.975755:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55cf5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.975763:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55cf5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22454272->22458367] (req 22454272->22458367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760757.975854:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508384896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34297us (34955us total) trans 4294972802 rc 0/0 00000100:00100000:1.0:1789760757.975864:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5514 00000100:00100000:1.0:1789760757.975874:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736989378543 time 737046253451 state 0x2 ref 0 00000001:00080000:3.0:1789760757.976534:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972802 is committed 00000100:00100000:3.0:1789760757.988745:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760757.988863:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508385024 00000100:00100000:1.0:1789760757.988872:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508385024 00000100:00100000:1.0:1789760757.988929:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 736989378543 time 737046253451 state 0x0 ref 1 00000100:00100000:1.0:1789760757.988944:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737046513867 time 737046253451 state 0x1 ref 1 00000100:00100000:1.0:1789760757.989030:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737046513867 time 737093740168 state 0x0 ref 1 00000100:00100000:1.0:1789760757.989044:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508385024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760757.989091:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22458368, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:1.0:1789760757.989100:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55cfc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.989115:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760757.989119:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760757.989125:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55cfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760757.989139:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55cfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.016337:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972803 is committed 00002000:00010000:3.0:1789760758.016497:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22458368 -> 22462464 00010000:00010000:3.0:1789760758.016513:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55cfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.016526:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55cfc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.016536:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55cfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.016544:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760758.016549:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55cfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.016559:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55cfc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.016578:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55cfc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.016586:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55cfc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22458368->22462463] (req 22458368->22462463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.017095:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508385024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28047us (28353us total) trans 4294972803 rc 0/0 00000100:00100000:3.0:1789760758.017106:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5515 00000100:00100000:3.0:1789760758.017116:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737046513867 time 737093740168 state 0x2 ref 0 00000100:00100000:2.0:1789760758.021257:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760758.023150:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508385152 00000100:00100000:3.0:1789760758.023165:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508385152 00000100:00100000:3.0:1789760758.023230:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737046513867 time 737093740168 state 0x0 ref 1 00000100:00100000:3.0:1789760758.023247:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737094000584 time 737093740168 state 0x1 ref 1 00000100:00100000:3.0:1789760758.023273:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737094000584 time 737127984380 state 0x0 ref 1 00000100:00100000:3.0:1789760758.023657:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508385152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760758.023713:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22462464, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:3.0:1789760758.023721:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.023738:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760758.023743:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.023749:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.023764:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.065866:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972804 is committed 00002000:00010000:3.0:1789760758.065867:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22462464 -> 22466560 00010000:00010000:3.0:1789760758.065885:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.065898:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.065907:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.065915:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760758.065920:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.065928:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.066072:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d03 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.066083:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22462464->22466559] (req 22462464->22466559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.066171:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508385152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42884us (44920us total) trans 4294972804 rc 0/0 00000100:00100000:3.0:1789760758.066343:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5516 00000100:00100000:3.0:1789760758.066354:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737094000584 time 737127984380 state 0x2 ref 0 00000100:00100000:1.0:1789760758.071278:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760758.071396:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508385280 00000100:00100000:2.0:1789760758.071406:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508385280 00000100:00100000:2.0:1789760758.071472:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737094000584 time 737127984380 state 0x0 ref 1 00000100:00100000:2.0:1789760758.071498:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737128244796 time 737127984380 state 0x1 ref 1 00000100:00100000:2.0:1789760758.071532:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737128244796 time 737176242195 state 0x0 ref 1 00000100:00100000:2.0:1789760758.071545:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508385280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760758.071595:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22466560, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:2.0:1789760758.071615:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55d0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.071635:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760758.071640:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.071646:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55d0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.071660:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55d0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760758.118989:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22466560 -> 22470656 00010000:00010000:1.0:1789760758.119011:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55d0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.119024:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55d0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.119033:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55d0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.119106:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760758.119112:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55d0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.119123:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55d0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.119124:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972805 is committed 00010000:00010000:1.0:1789760758.119151:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55d0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.119160:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe55d0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22466560->22470655] (req 22466560->22470655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.119336:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508385280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47787us (48062us total) trans 4294972805 rc 0/0 00000100:00100000:3.0:1789760758.119348:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5517 00000100:00100000:3.0:1789760758.119358:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737128244796 time 737176242195 state 0x2 ref 0 00000100:00100000:1.0:1789760758.122753:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760758.122850:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508385408 00000100:00100000:3.0:1789760758.122859:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508385408 00000100:00100000:3.0:1789760758.122919:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737128244796 time 737176242195 state 0x0 ref 1 00000100:00100000:3.0:1789760758.122935:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737176502611 time 737176242195 state 0x1 ref 1 00000100:00100000:3.0:1789760758.122973:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737176502611 time 737227683130 state 0x0 ref 1 00000100:00100000:3.0:1789760758.122986:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508385408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760758.123034:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22470656, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:3.0:1789760758.123042:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d11 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.123058:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760758.123062:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.123068:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.123083:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.159539:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972806 is committed 00002000:00010000:1.0:1789760758.159641:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22470656 -> 22474752 00010000:00010000:1.0:1789760758.159657:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.159670:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.159678:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.159685:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760758.159690:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.159698:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.159716:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.159724:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55d11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22470656->22474751] (req 22470656->22474751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.159820:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508385408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36831us (37073us total) trans 4294972806 rc 0/0 00000100:00100000:1.0:1789760758.159830:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5518 00000100:00100000:1.0:1789760758.159840:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737176502611 time 737227683130 state 0x2 ref 0 00000100:00100000:2.0:1789760758.163745:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760758.163911:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508385536 00000100:00100000:3.0:1789760758.163921:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508385536 00000100:00100000:3.0:1789760758.163987:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737176502611 time 737227683130 state 0x0 ref 1 00000100:00100000:3.0:1789760758.164002:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737227943546 time 737227683130 state 0x1 ref 1 00000100:00100000:3.0:1789760758.164065:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737227943546 time 737268774838 state 0x0 ref 1 00000100:00100000:3.0:1789760758.164078:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508385536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760758.164124:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22474752, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:3.0:1789760758.164138:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55d18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.164155:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760758.164160:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.164166:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55d18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.164184:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55d18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.186139:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972807 is committed 00002000:00010000:1.0:1789760758.186357:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22474752 -> 22478848 00010000:00010000:1.0:1789760758.186372:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55d18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.186386:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55d18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.186400:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55d18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.186408:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760758.186413:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55d18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.186423:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55d18 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.186444:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55d18 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.186453:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55d18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22474752->22478847] (req 22474752->22478847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.187072:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508385536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22990us (23329us total) trans 4294972807 rc 0/0 00000100:00100000:2.0:1789760758.187083:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5519 00000100:00100000:2.0:1789760758.187093:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737227943546 time 737268774838 state 0x2 ref 0 00000100:00100000:2.0:1789760758.192568:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760758.192652:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508385664 00000100:00100000:3.0:1789760758.192661:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508385664 00000100:00100000:3.0:1789760758.192717:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737227943546 time 737268774838 state 0x0 ref 1 00000100:00100000:3.0:1789760758.192734:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737269035254 time 737268774838 state 0x1 ref 1 00000100:00100000:3.0:1789760758.192763:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737269035254 time 737297473468 state 0x0 ref 1 00000100:00100000:3.0:1789760758.192775:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508385664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760758.192828:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22478848, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:3.0:1789760758.192838:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55d1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.192853:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760758.192857:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.192862:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55d1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.192877:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55d1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.227210:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972808 is committed 00002000:00010000:1.0:1789760758.227568:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22478848 -> 22482944 00010000:00010000:1.0:1789760758.227583:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55d1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.227597:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55d1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.227606:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55d1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.227619:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760758.227624:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55d1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.227633:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55d1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.227653:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55d1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.227660:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55d1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22478848->22482943] (req 22478848->22482943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.227753:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508385664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34975us (35189us total) trans 4294972808 rc 0/0 00000100:00100000:1.0:1789760758.227763:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5520 00000100:00100000:1.0:1789760758.227773:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737269035254 time 737297473468 state 0x2 ref 0 00000100:00100000:3.0:1789760758.231783:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760758.232206:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508385792 00000100:00100000:1.0:1789760758.232217:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508385792 00000100:00100000:1.0:1789760758.232269:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737269035254 time 737297473468 state 0x0 ref 1 00000100:00100000:1.0:1789760758.232723:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737297733884 time 737297473468 state 0x1 ref 1 00000100:00100000:0.0:1789760758.232923:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737297733884 time 737337631031 state 0x0 ref 1 00000100:00100000:0.0:1789760758.232939:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508385792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760758.233060:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22482944, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:0.0:1789760758.233071:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.233087:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.233091:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.233098:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.233113:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.265073:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972809 is committed 00002000:00010000:1.0:1789760758.265184:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22482944 -> 22487040 00010000:00010000:1.0:1789760758.265200:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.265213:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.265222:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.265230:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760758.265235:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.265245:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.265266:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.265274:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22482944->22487039] (req 22482944->22487039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.265407:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508385792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32467us (33629us total) trans 4294972809 rc 0/0 00000100:00100000:1.0:1789760758.265418:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5521 00000100:00100000:1.0:1789760758.265428:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737297733884 time 737337631031 state 0x2 ref 0 00000100:00100000:2.0:1789760758.268743:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760758.268855:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508385920 00000100:00100000:3.0:1789760758.268871:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508385920 00000100:00100000:3.0:1789760758.268925:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737297733884 time 737337631031 state 0x0 ref 1 00000100:00100000:3.0:1789760758.268940:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737337891447 time 737337631031 state 0x1 ref 1 00000100:00100000:3.0:1789760758.268980:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737337891447 time 737373689682 state 0x0 ref 1 00000100:00100000:3.0:1789760758.268992:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508385920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760758.269050:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22487040, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:3.0:1789760758.269059:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55d2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.269073:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760758.269078:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.269084:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55d2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.269098:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55d2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760758.303339:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22487040 -> 22491136 00010000:00010000:0.0:1789760758.303358:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55d2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.303372:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55d2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.303381:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55d2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.303389:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760758.303393:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55d2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.303404:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55d2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.303422:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55d2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.303430:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55d2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22487040->22491135] (req 22487040->22491135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.303515:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508385920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34520us (34777us total) trans 4294972810 rc 0/0 00000100:00100000:0.0:1789760758.303529:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5522 00000100:00100000:0.0:1789760758.303539:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737337891447 time 737373689682 state 0x2 ref 0 00000001:00080000:1.0:1789760758.303915:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972810 is committed 00000100:00100000:2.0:1789760758.307173:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760758.307254:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508386048 00000100:00100000:1.0:1789760758.307266:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508386048 00000100:00100000:1.0:1789760758.307338:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737337891447 time 737373689682 state 0x0 ref 1 00000100:00100000:1.0:1789760758.307354:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737373950098 time 737373689682 state 0x1 ref 1 00000100:00100000:1.0:1789760758.307388:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737373950098 time 737412098325 state 0x0 ref 1 00000100:00100000:1.0:1789760758.307401:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508386048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760758.307459:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22491136, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:1.0:1789760758.307469:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.307490:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.307495:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.307502:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.307517:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.333647:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972811 is committed 00002000:00010000:1.0:1789760758.333814:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22491136 -> 22495232 00010000:00010000:1.0:1789760758.333830:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.333844:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.333861:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.333869:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760758.333992:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.334004:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.334023:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.334031:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22491136->22495231] (req 22491136->22495231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.334152:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508386048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26749us (26984us total) trans 4294972811 rc 0/0 00000100:00100000:1.0:1789760758.334162:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5523 00000100:00100000:1.0:1789760758.334172:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737373950098 time 737412098325 state 0x2 ref 0 00000100:00100000:2.0:1789760758.350581:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760758.350826:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508386176 00000100:00100000:1.0:1789760758.350836:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508386176 00000100:00100000:1.0:1789760758.350891:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737373950098 time 737412098325 state 0x0 ref 1 00000100:00100000:1.0:1789760758.350907:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737412358741 time 737412098325 state 0x1 ref 1 00000100:00100000:1.0:1789760758.350939:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737412358741 time 737455649490 state 0x0 ref 1 00000100:00100000:1.0:1789760758.350962:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508386176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760758.351010:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22495232, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:1.0:1789760758.351020:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.351035:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.351040:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.351046:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.351060:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.390466:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972812 is committed 00002000:00010000:1.0:1789760758.390573:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22495232 -> 22499328 00010000:00010000:1.0:1789760758.390588:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.390602:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.390611:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.390619:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760758.390624:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.390633:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.390654:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.390661:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22495232->22499327] (req 22495232->22499327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.390789:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508386176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39834us (40214us total) trans 4294972812 rc 0/0 00000100:00100000:1.0:1789760758.390799:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5524 00000100:00100000:1.0:1789760758.390809:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737412358741 time 737455649490 state 0x2 ref 0 00000100:00100000:3.0:1789760758.398660:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760758.399469:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508386304 00000100:00100000:0.0:1789760758.399480:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508386304 00000100:00100000:0.0:1789760758.399546:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737412358741 time 737455649490 state 0x0 ref 1 00000100:00100000:0.0:1789760758.399561:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737455909906 time 737455649490 state 0x1 ref 1 00000100:00100000:0.0:1789760758.399596:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737455909906 time 737504306726 state 0x0 ref 1 00000100:00100000:0.0:1789760758.399610:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508386304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760758.399662:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22499328, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:0.0:1789760758.399672:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.399688:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.399692:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.399699:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.399712:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.435231:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972813 is committed 00002000:00010000:0.0:1789760758.435348:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22499328 -> 22503424 00010000:00010000:0.0:1789760758.435364:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.435377:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.435386:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.435394:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760758.435399:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.435407:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.435427:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.435436:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55d42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22499328->22503423] (req 22499328->22503423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.435532:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508386304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35920us (36876us total) trans 4294972813 rc 0/0 00000100:00100000:0.0:1789760758.435542:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5525 00000100:00100000:0.0:1789760758.435551:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737455909906 time 737504306726 state 0x2 ref 0 00000100:00100000:2.0:1789760758.448163:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760758.448321:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508386432 00000100:00100000:1.0:1789760758.448331:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508386432 00000100:00100000:1.0:1789760758.448458:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737455909906 time 737504306726 state 0x0 ref 1 00000100:00100000:1.0:1789760758.448473:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737504567142 time 737504306726 state 0x1 ref 1 00000100:00100000:1.0:1789760758.448536:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737504567142 time 737553246402 state 0x0 ref 1 00000100:00100000:1.0:1789760758.448548:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508386432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760758.448597:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22503424, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:1.0:1789760758.448606:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.448631:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.448637:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.448642:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.448657:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.481859:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972814 is committed 00002000:00010000:1.0:1789760758.481961:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22503424 -> 22507520 00010000:00010000:1.0:1789760758.481976:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.481990:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.482000:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.482008:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760758.482013:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.482021:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.482041:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.482050:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22503424->22507519] (req 22503424->22507519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.482176:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508386432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33624us (34018us total) trans 4294972814 rc 0/0 00000100:00100000:1.0:1789760758.482186:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5526 00000100:00100000:1.0:1789760758.482196:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737504567142 time 737553246402 state 0x2 ref 0 00000100:00100000:2.0:1789760758.485622:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760758.485835:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508386560 00000100:00100000:1.0:1789760758.485845:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508386560 00000100:00100000:1.0:1789760758.485908:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737504567142 time 737553246402 state 0x0 ref 1 00000100:00100000:1.0:1789760758.485923:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737553506818 time 737553246402 state 0x1 ref 1 00000100:00100000:1.0:1789760758.485959:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737553506818 time 737590669052 state 0x0 ref 1 00000100:00100000:1.0:1789760758.485972:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508386560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760758.486021:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22507520, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0xb007 00010000:00010000:1.0:1789760758.486029:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.486044:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.486049:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.486056:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.486070:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.517685:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972815 is committed 00002000:00010000:1.0:1789760758.517786:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22507520 -> 22511616 00002000:00010000:1.0:1789760758.517791:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760758 -> 1789760759 00002000:00010000:1.0:1789760758.517793:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760758 -> 1789760759 00010000:00010000:1.0:1789760758.517807:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.517820:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.517830:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.517837:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760758.517842:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.517851:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.517871:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.517880:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22507520->22511615] (req 22507520->22511615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.517982:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508386560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32007us (32365us total) trans 4294972815 rc 0/0 00000100:00100000:1.0:1789760758.517992:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5527 00000100:00100000:1.0:1789760758.518002:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737553506818 time 737590669052 state 0x2 ref 0 00000100:00100000:3.0:1789760758.521743:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760758.522179:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508386688 00000100:00100000:1.0:1789760758.522190:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508386688 00000100:00100000:1.0:1789760758.522244:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737553506818 time 737590669052 state 0x0 ref 1 00000100:00100000:1.0:1789760758.522259:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737590929468 time 737590669052 state 0x1 ref 1 00000100:00100000:1.0:1789760758.522331:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737590929468 time 737627040849 state 0x0 ref 1 00000100:00100000:1.0:1789760758.522344:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508386688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760758.522394:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22511616, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:1.0:1789760758.522409:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.522424:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.522428:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.522435:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.522453:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.532485:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760758.557665:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972816 is committed 00002000:00010000:0.0:1789760758.557768:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22511616 -> 22515712 00010000:00010000:0.0:1789760758.557784:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.557797:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.557807:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.557814:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760758.557819:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.557828:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.557849:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.557866:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22511616->22515711] (req 22511616->22515711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.557969:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508386688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35623us (36229us total) trans 4294972816 rc 0/0 00000100:00100000:0.0:1789760758.557980:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5528 00000100:00100000:0.0:1789760758.557990:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737590929468 time 737627040849 state 0x2 ref 0 00000100:00100000:2.0:1789760758.561801:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760758.561900:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508386816 00000100:00100000:1.0:1789760758.561909:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508386816 00000100:00100000:1.0:1789760758.561977:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737590929468 time 737627040849 state 0x0 ref 1 00000100:00100000:1.0:1789760758.561992:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737627301265 time 737627040849 state 0x1 ref 1 00000100:00100000:1.0:1789760758.562020:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737627301265 time 737666730744 state 0x0 ref 1 00000100:00100000:1.0:1789760758.562032:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508386816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760758.562081:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22515712, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:1.0:1789760758.562090:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.562105:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.562109:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.562116:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.562130:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760758.591210:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972817 is committed 00002000:00010000:3.0:1789760758.591793:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22515712 -> 22519808 00010000:00010000:3.0:1789760758.591809:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.592034:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.592047:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.592056:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760758.592060:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.592071:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.592092:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.592101:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22515712->22519807] (req 22515712->22519807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.592201:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508386816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30167us (30405us total) trans 4294972817 rc 0/0 00000100:00100000:3.0:1789760758.592212:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5529 00000100:00100000:3.0:1789760758.597370:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737627301265 time 737666730744 state 0x2 ref 0 00000100:00100000:2.0:1789760758.598837:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760758.599061:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508386944 00000100:00100000:0.0:1789760758.599072:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508386944 00000100:00100000:0.0:1789760758.599125:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737627301265 time 737666730744 state 0x0 ref 1 00000100:00100000:0.0:1789760758.599140:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737666991160 time 737666730744 state 0x1 ref 1 00000100:00100000:0.0:1789760758.599169:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737666991160 time 737703880152 state 0x0 ref 1 00000100:00100000:0.0:1789760758.599181:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508386944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760758.599227:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22519808, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:0.0:1789760758.599238:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.599254:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.599259:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.599266:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.599279:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760758.646844:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972818 is committed 00002000:00010000:0.0:1789760758.647088:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22519808 -> 22523904 00010000:00010000:0.0:1789760758.647103:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.647123:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.647132:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.647139:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760758.647144:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.647153:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.647173:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.647181:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22519808->22523903] (req 22519808->22523903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.647274:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508386944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48090us (48440us total) trans 4294972818 rc 0/0 00000100:00100000:0.0:1789760758.647327:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5530 00000100:00100000:0.0:1789760758.647338:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737666991160 time 737703880152 state 0x2 ref 0 00000100:00100000:0.0:1789760758.655513:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760758.656089:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508387072 00000100:00100000:1.0:1789760758.656099:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508387072 00000100:00100000:1.0:1789760758.656159:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737666991160 time 737703880152 state 0x0 ref 1 00000100:00100000:1.0:1789760758.656175:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737704140568 time 737703880152 state 0x1 ref 1 00000100:00100000:1.0:1789760758.656211:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737704140568 time 737760920848 state 0x0 ref 1 00000100:00100000:1.0:1789760758.656223:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508387072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760758.656273:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22523904, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:1.0:1789760758.656282:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d6c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.656362:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.656367:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.656373:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.658080:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760758.712710:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972819 is committed 00002000:00010000:1.0:1789760758.712797:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22523904 -> 22528000 00010000:00010000:1.0:1789760758.712812:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.712824:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.712839:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.712847:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760758.712857:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.712865:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.712885:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.712892:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55d6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22523904->22527999] (req 22523904->22527999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.712990:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508387072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56765us (57483us total) trans 4294972819 rc 0/0 00000100:00100000:1.0:1789760758.713000:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5531 00000100:00100000:1.0:1789760758.713009:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737704140568 time 737760920848 state 0x2 ref 0 00000100:00100000:2.0:1789760758.718212:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760758.718405:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508387200 00000100:00100000:0.0:1789760758.718416:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508387200 00000100:00100000:0.0:1789760758.718485:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737704140568 time 737760920848 state 0x0 ref 1 00000100:00100000:0.0:1789760758.718502:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737761181264 time 737760920848 state 0x1 ref 1 00000100:00100000:0.0:1789760758.718534:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737761181264 time 737823244468 state 0x0 ref 1 00000100:00100000:0.0:1789760758.718547:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508387200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760758.718597:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22528000, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:0.0:1789760758.718606:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.718622:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.718626:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.718633:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.718647:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760758.745457:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22528000 -> 22532096 00010000:00010000:2.0:1789760758.745478:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.745492:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.745502:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.745509:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760758.745514:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.745524:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.745545:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.745553:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55d73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22528000->22532095] (req 22528000->22532095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.745644:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508387200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27095us (27437us total) trans 4294972820 rc 0/0 00000100:00100000:2.0:1789760758.745655:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5532 00000100:00100000:2.0:1789760758.745664:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737761181264 time 737823244468 state 0x2 ref 0 00000001:00080000:3.0:1789760758.746015:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972820 is committed 00000100:00100000:2.0:1789760758.749645:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760758.753400:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508387328 00000100:00100000:0.0:1789760758.753413:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508387328 00000100:00100000:2.0:1789760758.754534:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737761181264 time 737823244468 state 0x0 ref 1 00000100:00100000:2.0:1789760758.754551:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737823504884 time 737823244468 state 0x1 ref 1 00000100:00100000:2.0:1789760758.754579:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737823504884 time 737859289079 state 0x0 ref 1 00000100:00100000:2.0:1789760758.754592:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508387328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760758.754641:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22532096, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:2.0:1789760758.754652:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55d7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.754667:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760758.754672:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.754678:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55d7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.754691:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55d7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760758.801018:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22532096 -> 22536192 00000001:00080000:3.0:1789760758.801023:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972821 is committed 00010000:00010000:2.0:1789760758.801040:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55d7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.801057:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55d7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.801067:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55d7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.801074:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760758.801078:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55d7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.801087:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55d7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.801107:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55d7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.801115:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55d7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22532096->22536191] (req 22532096->22536191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.801227:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508387328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46634us (51587us total) trans 4294972821 rc 0/0 00000100:00100000:2.0:1789760758.801241:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5533 00000100:00100000:2.0:1789760758.801250:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737823504884 time 737859289079 state 0x2 ref 0 00000100:00100000:2.0:1789760758.804193:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760758.804419:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508387456 00000100:00100000:1.0:1789760758.804429:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508387456 00000100:00100000:1.0:1789760758.804543:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737823504884 time 737859289079 state 0x0 ref 1 00000100:00100000:1.0:1789760758.804558:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737859549495 time 737859289079 state 0x1 ref 1 00000100:00100000:1.0:1789760758.804588:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737859549495 time 737909297899 state 0x0 ref 1 00000100:00100000:1.0:1789760758.804610:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508387456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760758.804658:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22536192, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:1.0:1789760758.804667:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.804682:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760758.804687:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.804693:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.804707:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760758.836273:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22536192 -> 22540288 00010000:00010000:1.0:1789760758.836334:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.836349:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.836359:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.836366:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760758.836373:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.836382:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.836405:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.836413:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55d81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22536192->22540287] (req 22536192->22540287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760758.836528:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508387456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31923us (32337us total) trans 4294972822 rc 0/0 00000100:00100000:1.0:1789760758.836540:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5534 00000100:00100000:1.0:1789760758.836551:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737859549495 time 737909297899 state 0x2 ref 0 00000001:00080000:3.0:1789760758.836816:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972822 is committed 00000100:00100000:1.0:1789760758.842862:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760758.843453:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508387584 00000100:00100000:2.0:1789760758.843468:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508387584 00000100:00100000:2.0:1789760758.843551:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737859549495 time 737909297899 state 0x0 ref 1 00000100:00100000:2.0:1789760758.843570:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737909558315 time 737909297899 state 0x1 ref 1 00000100:00100000:2.0:1789760758.843639:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737909558315 time 737948348073 state 0x0 ref 1 00000100:00100000:2.0:1789760758.843654:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508387584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760758.843730:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22540288, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:2.0:1789760758.843741:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55d88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.843759:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760758.843764:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.843771:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55d88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.843798:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55d88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.881312:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972823 is committed 00002000:00010000:2.0:1789760758.881553:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22540288 -> 22544384 00010000:00010000:2.0:1789760758.881573:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55d88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.881589:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55d88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.881600:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55d88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.881608:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760758.881614:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55d88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.881624:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55d88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.881649:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55d88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.881657:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe55d88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22540288->22544383] (req 22540288->22544383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.881836:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508387584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38178us (38982us total) trans 4294972823 rc 0/0 00000100:00100000:2.0:1789760758.881849:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5535 00000100:00100000:2.0:1789760758.881862:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737909558315 time 737948348073 state 0x2 ref 0 00000100:00100000:2.0:1789760758.884542:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760758.884679:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508387712 00000100:00100000:0.0:1789760758.884693:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508387712 00000100:00100000:0.0:1789760758.884764:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737909558315 time 737948348073 state 0x0 ref 1 00000100:00100000:0.0:1789760758.884783:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737948608489 time 737948348073 state 0x1 ref 1 00000100:00100000:0.0:1789760758.884817:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737948608489 time 737989526463 state 0x0 ref 1 00000100:00100000:0.0:1789760758.884831:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508387712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760758.884910:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22544384, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:0.0:1789760758.884921:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55d8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.884940:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760758.884957:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760758.884964:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55d8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760758.884980:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55d8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.929274:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972824 is committed 00002000:00010000:2.0:1789760758.929508:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22544384 -> 22548480 00010000:00010000:2.0:1789760758.929528:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55d8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.929544:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55d8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.929553:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55d8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.929560:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760758.929566:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55d8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.929576:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55d8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.929608:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55d8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.929617:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55d8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22544384->22548479] (req 22544384->22548479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760758.929802:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508387712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44965us (45266us total) trans 4294972824 rc 0/0 00000100:00100000:2.0:1789760758.929820:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5536 00000100:00100000:2.0:1789760758.929833:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737948608489 time 737989526463 state 0x2 ref 0 00000100:00100000:0.0:1789760758.933115:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760758.933221:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508387840 00000100:00100000:1.0:1789760758.933235:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508387840 00000100:00100000:1.0:1789760758.933474:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737948608489 time 737989526463 state 0x0 ref 1 00000100:00100000:1.0:1789760758.933494:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737989786879 time 737989526463 state 0x1 ref 1 00000100:00100000:1.0:1789760758.933563:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737989786879 time 738038272684 state 0x0 ref 1 00000100:00100000:1.0:1789760758.933584:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508387840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760758.933646:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22548480, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:1.0:1789760758.933657:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55d96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.933678:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760758.933683:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760758.933691:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55d96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.933706:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55d96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760758.964798:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972825 is committed 00002000:00010000:1.0:1789760758.965097:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22548480 -> 22552576 00010000:00010000:1.0:1789760758.965118:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55d96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.965135:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55d96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.965145:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55d96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.965153:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760758.965159:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55d96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.965171:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55d96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.965196:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55d96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760758.965205:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55d96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22548480->22552575] (req 22548480->22552575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760758.965574:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508387840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31985us (32463us total) trans 4294972825 rc 0/0 00000100:00100000:0.0:1789760758.965589:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5537 00000100:00100000:0.0:1789760758.965602:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737989786879 time 738038272684 state 0x2 ref 0 00000100:00100000:1.0:1789760758.968499:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760758.968593:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508387968 00000100:00100000:2.0:1789760758.968606:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508387968 00000100:00100000:2.0:1789760758.968678:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 737989786879 time 738038272684 state 0x0 ref 1 00000100:00100000:2.0:1789760758.968698:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738038533100 time 738038272684 state 0x1 ref 1 00000100:00100000:2.0:1789760758.968732:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738038533100 time 738073442883 state 0x0 ref 1 00000100:00100000:2.0:1789760758.968746:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508387968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760758.968805:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22552576, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:2.0:1789760758.968816:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55d9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.968835:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760758.968839:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760758.968847:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55d9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760758.968869:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55d9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.005690:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972826 is committed 00002000:00010000:2.0:1789760759.005864:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22552576 -> 22556672 00010000:00010000:2.0:1789760759.005885:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55d9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.005900:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55d9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.005911:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55d9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.005920:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760759.005925:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55d9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.005935:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55d9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.005969:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55d9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.005978:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55d9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22552576->22556671] (req 22552576->22556671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.006149:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508387968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37389us (37655us total) trans 4294972826 rc 0/0 00000100:00100000:2.0:1789760759.006162:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5538 00000100:00100000:2.0:1789760759.006176:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738038533100 time 738073442883 state 0x2 ref 0 00000100:00100000:3.0:1789760759.011189:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760759.011453:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508388096 00000100:00100000:2.0:1789760759.011467:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508388096 00000100:00100000:2.0:1789760759.011559:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738038533100 time 738073442883 state 0x0 ref 1 00000100:00100000:2.0:1789760759.011579:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738073703299 time 738073442883 state 0x1 ref 1 00000100:00100000:2.0:1789760759.011604:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738073703299 time 738116314226 state 0x0 ref 1 00000100:00100000:2.0:1789760759.011618:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508388096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760759.011681:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22556672, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:2.0:1789760759.011692:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55da4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.011712:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760759.011718:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00000100:00080000:1.0:1789760759.011725:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789760759.011725:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55da4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.011741:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55da4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.054816:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972827 is committed 00002000:00010000:2.0:1789760759.054902:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22556672 -> 22560768 00010000:00010000:2.0:1789760759.054923:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55da4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.054938:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55da4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.054965:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55da4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.055040:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760759.055047:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55da4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.055058:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55da4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.055085:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55da4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.055094:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe55da4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22556672->22560767] (req 22556672->22560767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.055260:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508388096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43636us (44080us total) trans 4294972827 rc 0/0 00000100:00100000:2.0:1789760759.055274:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5539 00000100:00100000:2.0:1789760759.055341:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738073703299 time 738116314226 state 0x2 ref 0 00000100:00100000:0.0:1789760759.058643:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760759.058715:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508388224 00000100:00100000:1.0:1789760759.058727:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508388224 00000100:00100000:1.0:1789760759.058802:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738073703299 time 738116314226 state 0x0 ref 1 00000100:00100000:1.0:1789760759.058820:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738116574642 time 738116314226 state 0x1 ref 1 00000100:00100000:1.0:1789760759.058928:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738116574642 time 738163637533 state 0x0 ref 1 00000100:00100000:1.0:1789760759.058955:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508388224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760759.059167:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22560768, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:1.0:1789760759.059178:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.059206:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.059211:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.059219:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.059236:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.109332:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972828 is committed 00002000:00010000:1.0:1789760759.109520:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22560768 -> 22564864 00010000:00010000:1.0:1789760759.109543:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.109560:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.109569:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.109577:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760759.109583:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.109593:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.109620:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.109628:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22560768->22564863] (req 22560768->22564863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.111599:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508388224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52644us (52953us total) trans 4294972828 rc 0/0 00000100:00100000:1.0:1789760759.111617:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5540 00000100:00100000:1.0:1789760759.111630:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738116574642 time 738163637533 state 0x2 ref 0 00000100:00100000:0.0:1789760759.117238:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760759.117616:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508388352 00000100:00100000:2.0:1789760759.117631:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508388352 00000100:00100000:2.0:1789760759.117718:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738116574642 time 738163637533 state 0x0 ref 1 00000100:00100000:2.0:1789760759.117742:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738163897949 time 738163637533 state 0x1 ref 1 00000100:00100000:2.0:1789760759.117778:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738163897949 time 738222488297 state 0x0 ref 1 00000100:00100000:2.0:1789760759.117792:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508388352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760759.117853:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22564864, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:2.0:1789760759.117874:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55db2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.117895:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760759.117900:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.117908:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55db2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.118035:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55db2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760759.169348:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22564864 -> 22568960 00010000:00010000:2.0:1789760759.169376:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55db2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.169393:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55db2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.169403:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55db2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.169417:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760759.169422:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55db2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.169434:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55db2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.169460:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55db2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.169469:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55db2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22564864->22568959] (req 22564864->22568959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.169582:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508388352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51785us (52352us total) trans 4294972829 rc 0/0 00000001:00080000:0.0:1789760759.169592:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972829 is committed 00000100:00100000:2.0:1789760759.169598:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5541 00000100:00100000:2.0:1789760759.169613:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738163897949 time 738222488297 state 0x2 ref 0 00000100:00100000:2.0:1789760759.174908:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760759.175162:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508388480 00000100:00100000:0.0:1789760759.175176:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508388480 00000100:00100000:0.0:1789760759.175255:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738163897949 time 738222488297 state 0x0 ref 1 00000100:00100000:0.0:1789760759.175274:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738222748713 time 738222488297 state 0x1 ref 1 00000100:00100000:0.0:1789760759.175578:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738222748713 time 738280286480 state 0x0 ref 1 00000100:00100000:0.0:1789760759.175595:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508388480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760759.175661:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22568960, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:0.0:1789760759.175684:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55db9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.175805:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760759.175811:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.175820:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55db9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.175836:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55db9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.225156:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972830 is committed 00002000:00010000:0.0:1789760759.225310:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22568960 -> 22573056 00010000:00010000:0.0:1789760759.225332:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55db9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.225347:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55db9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.225358:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55db9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.225369:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760759.225377:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55db9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.225388:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55db9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.225417:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55db9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.225424:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55db9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22568960->22573055] (req 22568960->22573055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.225791:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508388480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50191us (50889us total) trans 4294972830 rc 0/0 00000100:00100000:1.0:1789760759.225804:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5542 00000100:00100000:1.0:1789760759.225816:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738222748713 time 738280286480 state 0x2 ref 0 00000100:00100000:3.0:1789760759.234504:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760759.235138:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508388608 00000100:00100000:1.0:1789760759.235154:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508388608 00000100:00100000:1.0:1789760759.235231:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738222748713 time 738280286480 state 0x0 ref 1 00000100:00100000:1.0:1789760759.235252:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738280546896 time 738280286480 state 0x1 ref 1 00000100:00100000:1.0:1789760759.235539:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738280546896 time 738340247869 state 0x0 ref 1 00000100:00100000:1.0:1789760759.235557:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508388608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760759.235638:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22573056, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:1.0:1789760759.235650:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dc0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.235671:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.235676:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.235684:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.235700:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.294752:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972831 is committed 00002000:00010000:0.0:1789760759.294870:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22573056 -> 22577152 00010000:00010000:0.0:1789760759.294891:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.294905:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dc0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.294915:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.294922:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760759.294928:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.294938:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dc0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.294972:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dc0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.294980:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55dc0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22573056->22577151] (req 22573056->22577151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.295108:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508388608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59550us (60613us total) trans 4294972831 rc 0/0 00000100:00100000:0.0:1789760759.295119:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5543 00000100:00100000:0.0:1789760759.295130:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738280546896 time 738340247869 state 0x2 ref 0 00000100:00100000:0.0:1789760759.298856:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760759.299696:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508388736 00000100:00100000:2.0:1789760759.299709:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508388736 00000100:00100000:3.0:1789760759.299875:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738280546896 time 738340247869 state 0x0 ref 1 00000100:00100000:3.0:1789760759.299895:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738340508285 time 738340247869 state 0x1 ref 1 00000100:00100000:3.0:1789760759.300119:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738340508285 time 738404826843 state 0x0 ref 1 00000100:00100000:3.0:1789760759.300137:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508388736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760759.300201:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22577152, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:3.0:1789760759.300224:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55dc7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.300243:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760759.300248:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.300255:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55dc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.300271:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55dc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.347968:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972832 is committed 00002000:00010000:3.0:1789760759.348076:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22577152 -> 22581248 00010000:00010000:3.0:1789760759.348098:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55dc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.348120:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55dc7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.348130:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55dc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.348138:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760759.348144:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55dc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.348154:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55dc7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.348180:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55dc7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.348188:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55dc7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22577152->22581247] (req 22577152->22581247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.348578:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508388736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48437us (49728us total) trans 4294972832 rc 0/0 00000100:00100000:3.0:1789760759.348591:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5544 00000100:00100000:3.0:1789760759.348604:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738340508285 time 738404826843 state 0x2 ref 0 00000100:00100000:2.0:1789760759.351040:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760759.351413:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508388864 00000100:00100000:1.0:1789760759.351427:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508388864 00000100:00100000:1.0:1789760759.351566:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738340508285 time 738404826843 state 0x0 ref 1 00000100:00100000:1.0:1789760759.351585:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738405087259 time 738404826843 state 0x1 ref 1 00000100:00100000:1.0:1789760759.351670:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738405087259 time 738456379155 state 0x0 ref 1 00000100:00100000:1.0:1789760759.351685:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508388864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760759.351744:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22581248, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:1.0:1789760759.351755:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.351773:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.351778:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.351787:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.351810:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.401040:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22581248 -> 22585344 00000001:00080000:2.0:1789760759.401040:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972833 is committed 00010000:00010000:1.0:1789760759.401063:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.401079:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.401090:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.401099:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760759.401108:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.401119:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.401143:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.401152:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22581248->22585343] (req 22581248->22585343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.401267:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508388864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49577us (50233us total) trans 4294972833 rc 0/0 00000100:00100000:1.0:1789760759.401280:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5545 00000100:00100000:1.0:1789760759.401444:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738405087259 time 738456379155 state 0x2 ref 0 00000100:00100000:0.0:1789760759.403801:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760759.403936:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508388992 00000100:00100000:1.0:1789760759.403960:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508388992 00000100:00100000:1.0:1789760759.404041:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738405087259 time 738456379155 state 0x0 ref 1 00000100:00100000:1.0:1789760759.404062:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738456639571 time 738456379155 state 0x1 ref 1 00000100:00100000:1.0:1789760759.404099:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738456639571 time 738508809440 state 0x0 ref 1 00000100:00100000:1.0:1789760759.404113:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508388992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760759.404179:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22585344, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:1.0:1789760759.404189:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55dd5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.404210:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.404216:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.404224:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55dd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.404241:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55dd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760759.433504:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22585344 -> 22589440 00010000:00010000:1.0:1789760759.433534:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55dd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.433551:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55dd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.433561:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55dd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.433569:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760759.433575:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55dd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.433586:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55dd5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.433611:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55dd5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.433620:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55dd5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22585344->22589439] (req 22585344->22589439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760759.433741:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508388992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29623us (29946us total) trans 4294972834 rc 0/0 00000100:00100000:1.0:1789760759.433758:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5546 00000100:00100000:1.0:1789760759.433770:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738456639571 time 738508809440 state 0x2 ref 0 00000001:00080000:2.0:1789760759.435575:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972834 is committed 00000100:00100000:0.0:1789760759.437058:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760759.437178:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508389120 00000100:00100000:3.0:1789760759.437191:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508389120 00000100:00100000:3.0:1789760759.437258:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738456639571 time 738508809440 state 0x0 ref 1 00000100:00100000:3.0:1789760759.437277:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738509069856 time 738508809440 state 0x1 ref 1 00000100:00100000:3.0:1789760759.437809:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738509069856 time 738542515856 state 0x0 ref 1 00000100:00100000:3.0:1789760759.437827:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508389120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760759.437904:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22589440, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:3.0:1789760759.437916:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55ddc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.437936:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760759.437950:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.437957:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55ddc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.437976:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55ddc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760759.474343:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22589440 -> 22593536 00010000:00010000:0.0:1789760759.474372:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55ddc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.474388:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55ddc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.474398:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55ddc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.474407:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760759.474413:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55ddc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.474423:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55ddc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.474449:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55ddc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.474458:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55ddc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22589440->22593535] (req 22589440->22593535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.474591:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508389120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36761us (37536us total) trans 4294972835 rc 0/0 00000100:00100000:0.0:1789760759.474605:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5547 00000100:00100000:0.0:1789760759.474617:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738509069856 time 738542515856 state 0x2 ref 0 00000001:00080000:1.0:1789760759.474935:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972835 is committed 00000100:00100000:0.0:1789760759.479239:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760759.481949:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508389248 00000100:00100000:1.0:1789760759.481970:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508389248 00000100:00100000:1.0:1789760759.482375:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738509069856 time 738542515856 state 0x0 ref 1 00000100:00100000:1.0:1789760759.482396:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738542776272 time 738542515856 state 0x1 ref 1 00000100:00100000:1.0:1789760759.482445:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738542776272 time 738587155170 state 0x0 ref 1 00000100:00100000:1.0:1789760759.482462:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508389248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760759.482533:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22593536, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0xb007 00010000:00010000:1.0:1789760759.482544:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55de3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.482621:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.482628:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.482636:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55de3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.482656:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55de3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760759.519221:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972836 is committed 00002000:00010000:0.0:1789760759.519615:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22593536 -> 22597632 00002000:00010000:0.0:1789760759.519622:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760759 -> 1789760760 00002000:00010000:0.0:1789760759.519625:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760759 -> 1789760760 00010000:00010000:0.0:1789760759.519643:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55de3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.519677:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55de3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.519688:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55de3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.519696:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760759.519703:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55de3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.519713:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55de3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.519739:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55de3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.519747:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe55de3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22593536->22597631] (req 22593536->22597631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.519881:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508389248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37417us (40650us total) trans 4294972836 rc 0/0 00000100:00100000:0.0:1789760759.519894:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5548 00000100:00100000:0.0:1789760759.519907:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738542776272 time 738587155170 state 0x2 ref 0 00000100:00100000:2.0:1789760759.523652:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760759.523775:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508389376 00000100:00100000:1.0:1789760759.523788:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508389376 00000100:00100000:1.0:1789760759.523863:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738542776272 time 738587155170 state 0x0 ref 1 00000100:00100000:1.0:1789760759.523884:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738587415586 time 738587155170 state 0x1 ref 1 00000100:00100000:1.0:1789760759.523918:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738587415586 time 738628628135 state 0x0 ref 1 00000100:00100000:1.0:1789760759.523934:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508389376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760759.524020:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22597632, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:1.0:1789760759.524033:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.524052:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.524058:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.524066:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.524083:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.551172:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760759.564059:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22597632 -> 22601728 00010000:00010000:2.0:1789760759.564087:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.564104:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.564114:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.564123:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760759.564128:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.564140:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.564180:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.564190:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55dea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22597632->22601727] (req 22597632->22601727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.564427:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508389376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40489us (40782us total) trans 4294972837 rc 0/0 00000100:00100000:2.0:1789760759.564443:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5549 00000100:00100000:2.0:1789760759.564455:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738587415586 time 738628628135 state 0x2 ref 0 00000001:00080000:1.0:1789760759.564748:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972837 is committed 00000100:00100000:2.0:1789760759.574822:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760759.574965:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508389504 00000100:00100000:0.0:1789760759.574980:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508389504 00000100:00100000:0.0:1789760759.575120:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738587415586 time 738628628135 state 0x0 ref 1 00000100:00100000:0.0:1789760759.575140:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738628888551 time 738628628135 state 0x1 ref 1 00000100:00100000:0.0:1789760759.575212:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738628888551 time 738679920966 state 0x0 ref 1 00000100:00100000:0.0:1789760759.575227:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508389504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760759.575320:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22601728, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:0.0:1789760759.575331:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.575351:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760759.575356:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.575364:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.575381:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760759.611164:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22601728 -> 22605824 00010000:00010000:0.0:1789760759.611194:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.611211:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.611228:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.611237:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760759.611243:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.611258:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.611314:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.611325:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22601728->22605823] (req 22601728->22605823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.611469:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508389504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36224us (36641us total) trans 4294972838 rc 0/0 00000100:00100000:0.0:1789760759.611483:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5550 00000100:00100000:0.0:1789760759.611497:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738628888551 time 738679920966 state 0x2 ref 0 00000001:00080000:3.0:1789760759.612189:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972838 is committed 00000100:00100000:2.0:1789760759.617504:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760759.617625:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508389632 00000100:00100000:0.0:1789760759.617640:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508389632 00000100:00100000:0.0:1789760759.617779:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738628888551 time 738679920966 state 0x0 ref 1 00000100:00100000:0.0:1789760759.617798:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738680181382 time 738679920966 state 0x1 ref 1 00000100:00100000:0.0:1789760759.617872:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738680181382 time 738722580838 state 0x0 ref 1 00000100:00100000:0.0:1789760759.617888:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508389632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760759.617962:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22605824, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:0.0:1789760759.617973:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.618058:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760759.618063:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.618072:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.618090:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.661775:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972839 is committed 00002000:00010000:0.0:1789760759.661849:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22605824 -> 22609920 00010000:00010000:0.0:1789760759.661868:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.661882:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.661892:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.661900:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760759.661907:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.661918:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.661953:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.661962:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55df8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22605824->22609919] (req 22605824->22609919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.662090:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508389632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44198us (44595us total) trans 4294972839 rc 0/0 00000100:00100000:0.0:1789760759.662102:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5551 00000100:00100000:0.0:1789760759.662125:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738680181382 time 738722580838 state 0x2 ref 0 00000100:00100000:2.0:1789760759.664905:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760759.665234:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508389760 00000100:00100000:1.0:1789760759.665249:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508389760 00000100:00100000:1.0:1789760759.665462:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738680181382 time 738722580838 state 0x0 ref 1 00000100:00100000:1.0:1789760759.665481:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738722841254 time 738722580838 state 0x1 ref 1 00000100:00100000:1.0:1789760759.665556:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738722841254 time 738770265519 state 0x0 ref 1 00000100:00100000:1.0:1789760759.665572:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508389760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760759.665634:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22609920, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:1.0:1789760759.665646:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe55dff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.665666:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.665671:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.665679:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe55dff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.665696:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe55dff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.678486:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534077440:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760759.678587:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760759.678862:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534077440 00000100:00100000:1.0:1789760759.678873:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534077440 00000100:00100000:1.0:1789760759.678977:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 445 00000100:00100000:1.0:1789760759.678988:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534077440:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760759.679037:0:7358:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760759.679167:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534077440:12345-0@lo:41:osp-pre-1-0.0 Request processed in 180us (579us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760759.679175:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 445 00000100:00100000:2.0:1789760759.679356:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048846d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534077440:0@lo:41:osp-pre-1-0.0 00000001:00080000:3.0:1789760759.695390:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972840 is committed 00002000:00010000:2.0:1789760759.695666:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22609920 -> 22614016 00010000:00010000:2.0:1789760759.695686:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe55dff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.695700:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe55dff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.695710:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe55dff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.695717:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760759.695723:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe55dff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.695733:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe55dff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.695768:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe55dff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.695779:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe55dff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22609920->22614015] (req 22609920->22614015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.695913:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508389760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30336us (31015us total) trans 4294972840 rc 0/0 00000100:00100000:2.0:1789760759.695933:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5552 00000100:00100000:2.0:1789760759.695954:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738722841254 time 738770265519 state 0x2 ref 0 00000100:00100000:2.0:1789760759.699751:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760759.699864:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508389888 00000100:00100000:0.0:1789760759.699878:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508389888 00000100:00100000:0.0:1789760759.699971:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738722841254 time 738770265519 state 0x0 ref 1 00000100:00100000:0.0:1789760759.699992:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738770525935 time 738770265519 state 0x1 ref 1 00000100:00100000:0.0:1789760759.700025:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738770525935 time 738804734917 state 0x0 ref 1 00000100:00100000:0.0:1789760759.700043:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508389888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760759.700111:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22614016, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:0.0:1789760759.700124:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.700146:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760759.700151:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.700159:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.700176:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.735909:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972841 is committed 00002000:00010000:2.0:1789760759.736144:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22614016 -> 22618112 00010000:00010000:2.0:1789760759.736167:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.736182:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.736193:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.736202:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760759.736207:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.736216:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e06 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.736249:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e06 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.736258:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22614016->22618111] (req 22614016->22618111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.736440:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508389888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36392us (36696us total) trans 4294972841 rc 0/0 00000100:00100000:2.0:1789760759.736453:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5553 00000100:00100000:2.0:1789760759.736466:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738770525935 time 738804734917 state 0x2 ref 0 00000100:00100000:0.0:1789760759.742669:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760759.742770:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508390016 00000100:00100000:1.0:1789760759.742785:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508390016 00000100:00100000:1.0:1789760759.742864:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738770525935 time 738804734917 state 0x0 ref 1 00000100:00100000:1.0:1789760759.742884:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738804995333 time 738804734917 state 0x1 ref 1 00000100:00100000:1.0:1789760759.742914:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738804995333 time 738847624407 state 0x0 ref 1 00000100:00100000:1.0:1789760759.742928:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508390016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760759.743005:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22618112, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:1.0:1789760759.743017:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55e0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.743037:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760759.743042:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.743050:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55e0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.743067:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55e0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.788248:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972842 is committed 00002000:00010000:3.0:1789760759.788457:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22618112 -> 22622208 00010000:00010000:3.0:1789760759.788477:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55e0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.788499:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55e0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.788509:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55e0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.788517:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760759.788523:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55e0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.788532:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55e0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.788558:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55e0d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.788566:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55e0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22618112->22622207] (req 22618112->22622207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.788690:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508390016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45756us (46029us total) trans 4294972842 rc 0/0 00000100:00100000:3.0:1789760759.788702:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5554 00000100:00100000:3.0:1789760759.788714:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738804995333 time 738847624407 state 0x2 ref 0 00000100:00100000:3.0:1789760759.791402:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760759.791535:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508390144 00000100:00100000:2.0:1789760759.791548:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508390144 00000100:00100000:2.0:1789760759.791623:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738804995333 time 738847624407 state 0x0 ref 1 00000100:00100000:2.0:1789760759.791644:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738847884823 time 738847624407 state 0x1 ref 1 00000100:00100000:2.0:1789760759.791686:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738847884823 time 738896396090 state 0x0 ref 1 00000100:00100000:2.0:1789760759.791702:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508390144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760759.791769:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22622208, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:2.0:1789760759.791780:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55e14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.791807:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760759.791812:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760759.791820:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55e14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.791836:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55e14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760759.824918:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22622208 -> 22626304 00010000:00010000:2.0:1789760759.824956:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55e14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.824973:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55e14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.824983:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55e14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.824992:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760759.824999:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55e14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.825009:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55e14 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.825037:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55e14 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.825047:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe55e14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22622208->22626303] (req 22622208->22626303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.825174:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508390144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33468us (33777us total) trans 4294972843 rc 0/0 00000100:00100000:2.0:1789760759.825187:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5555 00000100:00100000:2.0:1789760759.825200:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738847884823 time 738896396090 state 0x2 ref 0 00000001:00080000:0.0:1789760759.825467:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972843 is committed 00000100:00100000:2.0:1789760759.828898:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760759.829119:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508390272 00000100:00100000:0.0:1789760759.829134:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508390272 00000100:00100000:0.0:1789760759.829220:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738847884823 time 738896396090 state 0x0 ref 1 00000100:00100000:0.0:1789760759.829241:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738896656506 time 738896396090 state 0x1 ref 1 00000100:00100000:0.0:1789760759.829281:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738896656506 time 738933990961 state 0x0 ref 1 00000100:00100000:0.0:1789760759.829446:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508390272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760759.829509:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22626304, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:0.0:1789760759.829521:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55e1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.829542:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760759.829546:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.829554:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55e1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.829571:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55e1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760759.882488:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22626304 -> 22630400 00010000:00010000:0.0:1789760759.882517:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55e1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.882533:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55e1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.882543:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55e1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.882551:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760759.882557:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55e1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.882568:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55e1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.882602:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55e1b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.882611:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe55e1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22626304->22630399] (req 22626304->22630399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760759.882877:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508390272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53428us (53986us total) trans 4294972844 rc 0/0 00000100:00100000:0.0:1789760759.882891:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5556 00000100:00100000:0.0:1789760759.882904:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738896656506 time 738933990961 state 0x2 ref 0 00000001:00080000:0.0:1789760759.882969:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972844 is committed 00000100:00100000:1.0:1789760759.886086:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760759.886171:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508390400 00000100:00100000:3.0:1789760759.886185:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508390400 00000100:00100000:3.0:1789760759.886262:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738896656506 time 738933990961 state 0x0 ref 1 00000100:00100000:3.0:1789760759.886281:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738934251377 time 738933990961 state 0x1 ref 1 00000100:00100000:3.0:1789760759.886335:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738934251377 time 738991045086 state 0x0 ref 1 00000100:00100000:3.0:1789760759.886348:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d40c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508390400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760759.886413:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22630400, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:3.0:1789760759.886424:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e22 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.886443:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760759.886448:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.886456:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.886473:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760759.923580:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972845 is committed 00002000:00010000:3.0:1789760759.923711:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22630400 -> 22634496 00010000:00010000:3.0:1789760759.923732:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.923749:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e22 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.923765:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.923772:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760759.923777:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.923787:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e22 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.923817:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e22 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.923827:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e22 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22630400->22634495] (req 22630400->22634495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.923963:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d40c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508390400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37611us (37884us total) trans 4294972845 rc 0/0 00000100:00100000:3.0:1789760759.923975:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5557 00000100:00100000:3.0:1789760759.923987:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738934251377 time 738991045086 state 0x2 ref 0 00000100:00100000:2.0:1789760759.928180:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760759.928255:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508390528 00000100:00100000:1.0:1789760759.928268:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508390528 00000100:00100000:1.0:1789760759.928369:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738934251377 time 738991045086 state 0x0 ref 1 00000100:00100000:1.0:1789760759.928389:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738991305502 time 738991045086 state 0x1 ref 1 00000100:00100000:1.0:1789760759.928429:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738991305502 time 739033138596 state 0x0 ref 1 00000100:00100000:1.0:1789760759.928450:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508390528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760759.928532:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22634496, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:1.0:1789760759.928543:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55e29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.928565:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760759.928570:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760759.928579:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55e29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760759.928597:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55e29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760759.991887:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972846 is committed 00002000:00010000:2.0:1789760759.992084:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22634496 -> 22638592 00010000:00010000:2.0:1789760759.992105:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55e29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.992121:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55e29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.992131:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55e29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.992140:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760759.992146:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55e29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.992163:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55e29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.992189:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55e29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760759.992197:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe55e29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22634496->22638591] (req 22634496->22638591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760759.993029:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508390528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 64570us (64849us total) trans 4294972846 rc 0/0 00000100:00100000:2.0:1789760759.993044:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5558 00000100:00100000:2.0:1789760759.993057:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738991305502 time 739033138596 state 0x2 ref 0 00000100:00100000:2.0:1789760759.998664:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760759.998787:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508390656 00000100:00100000:0.0:1789760759.998800:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508390656 00000100:00100000:0.0:1789760759.998867:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 738991305502 time 739033138596 state 0x0 ref 1 00000100:00100000:0.0:1789760759.998884:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739033399012 time 739033138596 state 0x1 ref 1 00000100:00100000:0.0:1789760759.998916:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739033399012 time 739103626267 state 0x0 ref 1 00000100:00100000:0.0:1789760759.998928:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508390656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760759.999002:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22638592, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:0.0:1789760759.999013:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.999038:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760759.999043:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760759.999050:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760759.999067:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.050799:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972847 is committed 00002000:00010000:0.0:1789760760.052544:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22638592 -> 22642688 00010000:00010000:0.0:1789760760.052570:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.052586:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.052596:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.052604:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.052609:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.052619:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.052645:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e30 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.052655:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22638592->22642687] (req 22638592->22642687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.052799:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508390656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53865us (54143us total) trans 4294972847 rc 0/0 00000100:00100000:0.0:1789760760.052811:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5559 00000100:00100000:0.0:1789760760.052824:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739033399012 time 739103626267 state 0x2 ref 0 00000100:00100000:0.0:1789760760.056733:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760760.056915:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508390784 00000100:00100000:3.0:1789760760.056929:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508390784 00000100:00100000:3.0:1789760760.057076:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739033399012 time 739103626267 state 0x0 ref 1 00000100:00100000:3.0:1789760760.057095:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739103886683 time 739103626267 state 0x1 ref 1 00000100:00100000:3.0:1789760760.057176:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739103886683 time 739161885141 state 0x0 ref 1 00000100:00100000:3.0:1789760760.057190:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508390784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760760.057255:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22642688, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:3.0:1789760760.057265:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.057318:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.057324:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.057332:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.057349:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.120700:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972848 is committed 00002000:00010000:0.0:1789760760.120813:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22642688 -> 22646784 00010000:00010000:0.0:1789760760.120832:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.120847:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.120858:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.120866:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.120871:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.120882:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e37 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.120916:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e37 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.120926:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe55e37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22642688->22646783] (req 22642688->22646783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.121073:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508390784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 63877us (64347us total) trans 4294972848 rc 0/0 00000100:00100000:0.0:1789760760.121086:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5560 00000100:00100000:0.0:1789760760.121099:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739103886683 time 739161885141 state 0x2 ref 0 00000100:00100000:2.0:1789760760.124689:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760760.124809:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508390912 00000100:00100000:3.0:1789760760.124821:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508390912 00000100:00100000:3.0:1789760760.125041:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739103886683 time 739161885141 state 0x0 ref 1 00000100:00100000:3.0:1789760760.125066:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739162145557 time 739161885141 state 0x1 ref 1 00000100:00100000:3.0:1789760760.125100:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739162145557 time 739229809975 state 0x0 ref 1 00000100:00100000:3.0:1789760760.125115:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508390912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760760.125179:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22646784, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:3.0:1789760760.125192:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55e3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.125212:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.125218:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.125226:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55e3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.125244:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55e3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.160075:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972849 is committed 00002000:00010000:0.0:1789760760.160512:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22646784 -> 22650880 00010000:00010000:0.0:1789760760.160535:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55e3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.160550:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55e3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.160561:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55e3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.160569:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.160673:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55e3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.160691:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55e3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.160716:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55e3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.160725:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe55e3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22646784->22650879] (req 22646784->22650879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.161123:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508390912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36003us (36441us total) trans 4294972849 rc 0/0 00000100:00100000:0.0:1789760760.161135:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5561 00000100:00100000:0.0:1789760760.161147:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739162145557 time 739229809975 state 0x2 ref 0 00000100:00100000:2.0:1789760760.167787:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760760.167908:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508391040 00000100:00100000:0.0:1789760760.167929:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508391040 00000100:00100000:0.0:1789760760.168016:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739162145557 time 739229809975 state 0x0 ref 1 00000100:00100000:0.0:1789760760.168035:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739230070391 time 739229809975 state 0x1 ref 1 00000100:00100000:0.0:1789760760.168069:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739230070391 time 739272779243 state 0x0 ref 1 00000100:00100000:0.0:1789760760.168083:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508391040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760760.168158:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22650880, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:0.0:1789760760.168169:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.168192:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.168197:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.168205:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.168222:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.231827:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972850 is committed 00002000:00010000:0.0:1789760760.231866:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22650880 -> 22654976 00010000:00010000:0.0:1789760760.231890:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.231907:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.231917:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.231925:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.231930:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.231951:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e45 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.231976:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e45 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.231985:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22650880->22654975] (req 22650880->22654975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.232105:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508391040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 64017us (64326us total) trans 4294972850 rc 0/0 00000100:00100000:0.0:1789760760.232117:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5562 00000100:00100000:0.0:1789760760.232131:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739230070391 time 739272779243 state 0x2 ref 0 00000100:00100000:1.0:1789760760.240180:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760760.240322:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508391168 00000100:00100000:0.0:1789760760.240336:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508391168 00000100:00100000:0.0:1789760760.240405:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739230070391 time 739272779243 state 0x0 ref 1 00000100:00100000:0.0:1789760760.240424:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739273039659 time 739272779243 state 0x1 ref 1 00000100:00100000:0.0:1789760760.240525:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739273039659 time 739345233555 state 0x0 ref 1 00000100:00100000:0.0:1789760760.240540:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508391168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760760.240608:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22654976, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:0.0:1789760760.240619:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.240639:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.240643:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.240651:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.240673:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.302145:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972851 is committed 00002000:00010000:0.0:1789760760.302146:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22654976 -> 22659072 00010000:00010000:0.0:1789760760.302163:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.302175:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.302184:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.302191:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.302196:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.302205:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.302225:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.302233:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22654976->22659071] (req 22654976->22659071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.302367:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508391168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61826us (62199us total) trans 4294972851 rc 0/0 00000100:00100000:0.0:1789760760.302389:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5563 00000100:00100000:0.0:1789760760.302399:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739273039659 time 739345233555 state 0x2 ref 0 00000100:00100000:2.0:1789760760.309174:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760760.309279:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508391296 00000100:00100000:3.0:1789760760.309306:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508391296 00000100:00100000:3.0:1789760760.309363:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739273039659 time 739345233555 state 0x0 ref 1 00000100:00100000:3.0:1789760760.309513:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739345493971 time 739345233555 state 0x1 ref 1 00000100:00100000:3.0:1789760760.309566:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739345493971 time 739414276276 state 0x0 ref 1 00000100:00100000:3.0:1789760760.309576:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508391296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760760.309614:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22659072, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:3.0:1789760760.309621:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55e53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.309633:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.309636:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.309641:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55e53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.309651:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55e53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.356715:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972852 is committed 00002000:00010000:0.0:1789760760.356743:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22659072 -> 22663168 00010000:00010000:0.0:1789760760.356758:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55e53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.356770:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55e53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.356779:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55e53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.356786:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.356790:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55e53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.356798:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55e53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.356818:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55e53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.356831:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe55e53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22659072->22663167] (req 22659072->22663167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.356944:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508391296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47355us (47767us total) trans 4294972852 rc 0/0 00000100:00100000:0.0:1789760760.356955:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5564 00000100:00100000:0.0:1789760760.356965:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739345493971 time 739414276276 state 0x2 ref 0 00000100:00100000:1.0:1789760760.364758:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760760.366020:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508391424 00000100:00100000:0.0:1789760760.366033:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508391424 00000100:00100000:0.0:1789760760.366090:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739345493971 time 739414276276 state 0x0 ref 1 00000100:00100000:0.0:1789760760.366104:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739414536692 time 739414276276 state 0x1 ref 1 00000100:00100000:0.0:1789760760.366141:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739414536692 time 739470850860 state 0x0 ref 1 00000100:00100000:0.0:1789760760.366154:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508391424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760760.366199:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22663168, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:0.0:1789760760.366209:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e5a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.366224:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.366229:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.366235:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.366254:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.408760:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972853 is committed 00002000:00010000:0.0:1789760760.408917:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22663168 -> 22667264 00010000:00010000:0.0:1789760760.408939:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.408952:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.408961:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.408968:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.408973:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.408982:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.409001:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.409008:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22663168->22667263] (req 22663168->22667263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.409573:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508391424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43416us (44818us total) trans 4294972853 rc 0/0 00000100:00100000:0.0:1789760760.409583:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5565 00000100:00100000:0.0:1789760760.409592:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739414536692 time 739470850860 state 0x2 ref 0 00000100:00100000:1.0:1789760760.419834:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760760.419897:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508391552 00000100:00100000:0.0:1789760760.419906:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508391552 00000100:00100000:0.0:1789760760.419965:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739414536692 time 739470850860 state 0x0 ref 1 00000100:00100000:0.0:1789760760.419980:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739471111276 time 739470850860 state 0x1 ref 1 00000100:00100000:0.0:1789760760.420010:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739471111276 time 739524720287 state 0x0 ref 1 00000100:00100000:0.0:1789760760.420022:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508391552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760760.420089:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22667264, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:0.0:1789760760.420099:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.420114:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.420118:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.420124:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.420137:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760760.466853:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22667264 -> 22671360 00010000:00010000:0.0:1789760760.466874:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.466887:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.466897:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.466903:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.466908:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.466916:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.466947:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.466955:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22667264->22671359] (req 22667264->22671359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.467051:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508391552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47025us (47217us total) trans 4294972854 rc 0/0 00000100:00100000:0.0:1789760760.467066:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5566 00000100:00100000:0.0:1789760760.467077:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739471111276 time 739524720287 state 0x2 ref 0 00000001:00080000:3.0:1789760760.467619:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972854 is committed 00000100:00100000:3.0:1789760760.472697:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760760.472774:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508391680 00000100:00100000:0.0:1789760760.472787:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508391680 00000100:00100000:0.0:1789760760.472839:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739471111276 time 739524720287 state 0x0 ref 1 00000100:00100000:0.0:1789760760.472854:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739524980703 time 739524720287 state 0x1 ref 1 00000100:00100000:0.0:1789760760.472886:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739524980703 time 739577596359 state 0x0 ref 1 00000100:00100000:0.0:1789760760.472898:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508391680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760760.472954:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22671360, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0xb007 00010000:00010000:0.0:1789760760.472964:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.472985:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.472989:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.472997:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.473013:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.497321:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972855 is committed 00002000:00010000:0.0:1789760760.497538:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22671360 -> 22675456 00002000:00010000:0.0:1789760760.497543:0:14638:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760760 -> 1789760761 00002000:00010000:0.0:1789760760.497546:0:14638:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760760 -> 1789760761 00010000:00010000:0.0:1789760760.497559:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.497572:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.497581:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.497588:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.497592:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.497600:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.497619:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.497627:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22671360->22675455] (req 22671360->22675455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.497757:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508391680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24855us (25064us total) trans 4294972855 rc 0/0 00000100:00100000:0.0:1789760760.497768:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5567 00000100:00100000:0.0:1789760760.497779:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739524980703 time 739577596359 state 0x2 ref 0 00000100:00100000:0.0:1789760760.507840:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760760.508079:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508391808 00000100:00100000:2.0:1789760760.508097:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508391808 00000100:00100000:2.0:1789760760.508788:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739524980703 time 739577596359 state 0x0 ref 1 00000100:00100000:2.0:1789760760.508805:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739577856775 time 739577596359 state 0x1 ref 1 00000100:00100000:2.0:1789760760.508897:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739577856775 time 739613606709 state 0x0 ref 1 00000100:00100000:2.0:1789760760.508913:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508391808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760760.508972:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22675456, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:2.0:1789760760.508981:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55e6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.508997:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760760.509002:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760760.509008:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55e6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760760.509023:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55e6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760760.549594:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972856 is committed 00002000:00010000:0.0:1789760760.549692:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22675456 -> 22679552 00010000:00010000:0.0:1789760760.549708:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55e6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.549720:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55e6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.549729:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55e6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.549737:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.549742:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55e6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.549750:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55e6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.549769:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55e6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.549777:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55e6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22675456->22679551] (req 22675456->22679551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.549869:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508391808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40956us (42035us total) trans 4294972856 rc 0/0 00000100:00100000:0.0:1789760760.549893:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5568 00000100:00100000:0.0:1789760760.549903:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739577856775 time 739613606709 state 0x2 ref 0 00000100:00100000:1.0:1789760760.554398:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760760.554510:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508391936 00000100:00100000:0.0:1789760760.554520:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508391936 00000100:00100000:0.0:1789760760.554577:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739577856775 time 739613606709 state 0x0 ref 1 00000100:00100000:0.0:1789760760.554593:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739613867125 time 739613606709 state 0x1 ref 1 00000100:00100000:0.0:1789760760.554618:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739613867125 time 739659328524 state 0x0 ref 1 00000100:00100000:0.0:1789760760.554630:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508391936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760760.554686:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22679552, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:0.0:1789760760.554695:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e76 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.554710:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.554715:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.554721:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.554735:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.575982:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760760.594077:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972857 is committed 00002000:00010000:0.0:1789760760.594179:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22679552 -> 22683648 00010000:00010000:0.0:1789760760.594196:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.594210:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e76 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.594220:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.594228:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.594233:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.594243:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e76 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.594262:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e76 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.594269:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e76 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22679552->22683647] (req 22679552->22683647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.594722:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508391936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40087us (40328us total) trans 4294972857 rc 0/0 00000100:00100000:0.0:1789760760.594733:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5569 00000100:00100000:0.0:1789760760.594742:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739613867125 time 739659328524 state 0x2 ref 0 00000100:00100000:3.0:1789760760.599483:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760760.599697:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508392064 00000100:00100000:0.0:1789760760.599708:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508392064 00000100:00100000:0.0:1789760760.599767:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739613867125 time 739659328524 state 0x0 ref 1 00000100:00100000:0.0:1789760760.599782:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739659588940 time 739659328524 state 0x1 ref 1 00000100:00100000:0.0:1789760760.599813:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739659588940 time 739704523158 state 0x0 ref 1 00000100:00100000:0.0:1789760760.599988:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508392064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760760.600035:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22683648, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:0.0:1789760760.600043:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55e7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.600059:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.600063:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.600070:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55e7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.600084:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55e7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.625888:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972858 is committed 00002000:00010000:3.0:1789760760.625980:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22683648 -> 22687744 00010000:00010000:3.0:1789760760.625995:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55e7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.626008:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55e7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.626017:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55e7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.626024:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760760.626029:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55e7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.626039:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55e7d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.626063:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55e7d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.626071:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe55e7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22683648->22687743] (req 22683648->22687743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.626189:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508392064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26361us (26711us total) trans 4294972858 rc 0/0 00000100:00100000:3.0:1789760760.626200:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5570 00000100:00100000:3.0:1789760760.626209:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739659588940 time 739704523158 state 0x2 ref 0 00000100:00100000:2.0:1789760760.629632:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760760.629730:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508392192 00000100:00100000:0.0:1789760760.629740:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508392192 00000100:00100000:0.0:1789760760.629793:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739659588940 time 739704523158 state 0x0 ref 1 00000100:00100000:0.0:1789760760.629808:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739704783574 time 739704523158 state 0x1 ref 1 00000100:00100000:0.0:1789760760.629837:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739704783574 time 739734547753 state 0x0 ref 1 00000100:00100000:0.0:1789760760.629861:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508392192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760760.629907:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22687744, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:0.0:1789760760.629917:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.629941:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.629945:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.629952:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.629966:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.659874:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972859 is committed 00002000:00010000:0.0:1789760760.660045:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22687744 -> 22691840 00010000:00010000:0.0:1789760760.660059:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.660072:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.660082:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.660089:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.660093:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.660103:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.660123:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.660131:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22687744->22691839] (req 22687744->22691839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.660220:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508392192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30358us (30593us total) trans 4294972859 rc 0/0 00000100:00100000:0.0:1789760760.660230:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5571 00000100:00100000:0.0:1789760760.660240:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739704783574 time 739734547753 state 0x2 ref 0 00000100:00100000:3.0:1789760760.663494:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760760.663611:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508392320 00000100:00100000:0.0:1789760760.663620:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508392320 00000100:00100000:0.0:1789760760.663673:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739704783574 time 739734547753 state 0x0 ref 1 00000100:00100000:0.0:1789760760.663687:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739734808169 time 739734547753 state 0x1 ref 1 00000100:00100000:0.0:1789760760.663710:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739734808169 time 739768421491 state 0x0 ref 1 00000100:00100000:0.0:1789760760.663722:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508392320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760760.663770:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22691840, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:0.0:1789760760.663779:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.663796:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.663801:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.663807:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.663821:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760760.694565:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22691840 -> 22695936 00010000:00010000:0.0:1789760760.694585:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.694598:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.694607:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.694615:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.694619:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.694628:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.694648:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.694657:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22691840->22695935] (req 22691840->22695935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.694751:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508392320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31024us (31260us total) trans 4294972860 rc 0/0 00000100:00100000:0.0:1789760760.694762:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5572 00000100:00100000:0.0:1789760760.694775:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739734808169 time 739768421491 state 0x2 ref 0 00000001:00080000:3.0:1789760760.694864:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972860 is committed 00000100:00100000:3.0:1789760760.698413:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760760.698626:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508392448 00000100:00100000:0.0:1789760760.698635:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508392448 00000100:00100000:0.0:1789760760.698695:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739734808169 time 739768421491 state 0x0 ref 1 00000100:00100000:0.0:1789760760.698711:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739768681907 time 739768421491 state 0x1 ref 1 00000100:00100000:0.0:1789760760.698736:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739768681907 time 739803446893 state 0x0 ref 1 00000100:00100000:0.0:1789760760.698748:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508392448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760760.698796:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22695936, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:0.0:1789760760.698805:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.698834:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.698839:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.698850:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.698868:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.727173:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972861 is committed 00002000:00010000:0.0:1789760760.727275:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22695936 -> 22700032 00010000:00010000:0.0:1789760760.727380:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.727393:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.727402:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.727409:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.727414:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.727423:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.727443:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.727451:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe55e92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22695936->22700031] (req 22695936->22700031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.727557:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508392448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28806us (29152us total) trans 4294972861 rc 0/0 00000100:00100000:0.0:1789760760.727567:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5573 00000100:00100000:0.0:1789760760.727577:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739768681907 time 739803446893 state 0x2 ref 0 00000100:00100000:2.0:1789760760.730517:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760760.730633:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508392576 00000100:00100000:0.0:1789760760.730643:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508392576 00000100:00100000:0.0:1789760760.730693:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739768681907 time 739803446893 state 0x0 ref 1 00000100:00100000:0.0:1789760760.730708:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739803707309 time 739803446893 state 0x1 ref 1 00000100:00100000:0.0:1789760760.730728:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739803707309 time 739835439091 state 0x0 ref 1 00000100:00100000:0.0:1789760760.730740:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508392576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760760.730786:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22700032, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:0.0:1789760760.730795:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.730810:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760760.730814:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760760.730820:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.730832:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760760.758916:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22700032 -> 22704128 00010000:00010000:0.0:1789760760.758945:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.758959:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.758969:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.758977:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760760.758981:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.758990:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.759007:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760760.759014:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe55e99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22700032->22704127] (req 22700032->22704127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760760.759107:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508392576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28364us (28593us total) trans 4294972862 rc 0/0 00000100:00100000:0.0:1789760760.759121:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5574 00000100:00100000:0.0:1789760760.759130:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739803707309 time 739835439091 state 0x2 ref 0 00000001:00080000:3.0:1789760760.762239:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972862 is committed 00000100:00100000:0.0:1789760760.762715:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760760.762786:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508392704 00000100:00100000:3.0:1789760760.762795:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508392704 00000100:00100000:3.0:1789760760.762853:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739803707309 time 739835439091 state 0x0 ref 1 00000100:00100000:3.0:1789760760.762868:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739835699507 time 739835439091 state 0x1 ref 1 00000100:00100000:3.0:1789760760.762893:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739835699507 time 739867604193 state 0x0 ref 1 00000100:00100000:3.0:1789760760.762905:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508392704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760760.762959:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22704128, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:3.0:1789760760.762967:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55ea0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.762983:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.762988:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.762994:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55ea0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.763007:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55ea0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.794116:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22704128 -> 22708224 00010000:00010000:3.0:1789760760.794134:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55ea0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.794147:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55ea0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.794156:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55ea0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.794164:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760760.794169:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55ea0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.794177:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55ea0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.794197:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55ea0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.794205:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55ea0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22704128->22708223] (req 22704128->22708223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.794333:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508392704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31425us (31619us total) trans 4294972863 rc 0/0 00000100:00100000:3.0:1789760760.794344:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5575 00000100:00100000:3.0:1789760760.794353:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739835699507 time 739867604193 state 0x2 ref 0 00000001:00080000:3.0:1789760760.794388:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972863 is committed 00000100:00100000:2.0:1789760760.797146:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760760.797429:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508392832 00000100:00100000:3.0:1789760760.797439:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508392832 00000100:00100000:3.0:1789760760.797495:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739835699507 time 739867604193 state 0x0 ref 1 00000100:00100000:3.0:1789760760.797510:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739867864609 time 739867604193 state 0x1 ref 1 00000100:00100000:3.0:1789760760.797537:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739867864609 time 739902247998 state 0x0 ref 1 00000100:00100000:3.0:1789760760.797549:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508392832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760760.797592:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22708224, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:3.0:1789760760.797600:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55ea7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.797621:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.797625:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.797631:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55ea7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.797645:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55ea7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760760.831353:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22708224 -> 22712320 00010000:00010000:1.0:1789760760.831384:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55ea7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.831397:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55ea7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.831412:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55ea7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.831419:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760760.831424:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55ea7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.831434:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55ea7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.831452:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55ea7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.831461:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe55ea7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22708224->22712319] (req 22708224->22712319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.831547:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508392832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33996us (34406us total) trans 4294972864 rc 0/0 00000100:00100000:1.0:1789760760.831557:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5576 00000100:00100000:1.0:1789760760.831566:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739867864609 time 739902247998 state 0x2 ref 0 00000001:00080000:2.0:1789760760.832735:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972864 is committed 00000100:00100000:2.0:1789760760.835233:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760760.835513:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508392960 00000100:00100000:0.0:1789760760.835525:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508392960 00000100:00100000:3.0:1789760760.835648:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739867864609 time 739902247998 state 0x0 ref 1 00000100:00100000:3.0:1789760760.835663:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739902508414 time 739902247998 state 0x1 ref 1 00000100:00100000:3.0:1789760760.835719:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739902508414 time 739940429330 state 0x0 ref 1 00000100:00100000:3.0:1789760760.835731:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508392960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760760.835778:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22712320, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:3.0:1789760760.835787:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55eae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.835803:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.835807:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.835813:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55eae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.835827:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55eae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760760.861228:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972865 is committed 00002000:00010000:1.0:1789760760.861350:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22712320 -> 22716416 00010000:00010000:1.0:1789760760.861366:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55eae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.861385:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55eae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.861394:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55eae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.861401:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760760.861405:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55eae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.861414:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55eae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.861433:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55eae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.861441:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe55eae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22712320->22716415] (req 22712320->22716415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.861527:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508392960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25793us (26299us total) trans 4294972865 rc 0/0 00000100:00100000:1.0:1789760760.861537:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5577 00000100:00100000:1.0:1789760760.861546:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739902508414 time 739940429330 state 0x2 ref 0 00000100:00100000:0.0:1789760760.865950:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760760.866084:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508393088 00000100:00100000:3.0:1789760760.866094:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508393088 00000100:00100000:3.0:1789760760.866145:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739902508414 time 739940429330 state 0x0 ref 1 00000100:00100000:3.0:1789760760.866159:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739940689746 time 739940429330 state 0x1 ref 1 00000100:00100000:3.0:1789760760.866230:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739940689746 time 739970940096 state 0x0 ref 1 00000100:00100000:3.0:1789760760.866243:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508393088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760760.866319:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22716416, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:3.0:1789760760.866328:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55eb5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.866343:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760760.866347:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.866354:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55eb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.866369:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55eb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.897671:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760760.897703:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760760.897729:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760760.897748:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760760.897769:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760760.897796:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508393344 00000100:00100000:3.0:1789760760.897805:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508393344 00000100:00100000:3.0:1789760760.897870:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508393600 00000100:00100000:3.0:1789760760.897875:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508393600 00000100:00100000:3.0:1789760760.897893:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508393472 00000100:00100000:3.0:1789760760.897896:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508393472 00000100:00100000:3.0:1789760760.897907:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508393728 00000100:00100000:3.0:1789760760.897909:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508393728 00000100:00100000:3.0:1789760760.897922:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 929 00000100:00100000:3.0:1789760760.897941:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2409:x1876699508393344:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:3.0:1789760760.898008:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2409:x1876699508393344:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 79us (346us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760760.898016:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 929 00000100:00100000:3.0:1789760760.898031:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 930 00000100:00100000:3.0:1789760760.898036:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508393600:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:3.0:1789760760.898055:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508393600:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 19us (352us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760760.898061:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 930 00000100:00100000:3.0:1789760760.898069:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 931 00000100:00100000:3.0:1789760760.898075:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508393472:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:3.0:1789760760.898092:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508393472:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 17us (344us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760760.898097:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 931 00000100:00100000:3.0:1789760760.898105:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 932 00000100:00100000:3.0:1789760760.898110:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508393728:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:3.0:1789760760.898127:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508393728:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 17us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760760.898133:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 932 00000100:00100000:1.0:1789760760.899986:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508393216 00000100:00100000:1.0:1789760760.899999:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508393216 00000100:00100000:1.0:1789760760.900076:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 479 00000100:00100000:1.0:1789760760.900086:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cad050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2409:x1876699508393216:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760760.900161:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cad050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2409:x1876699508393216:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 76us (2431us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760760.900169:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 479 00000100:00080000:1.0:1789760760.900201:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760760.900206:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789760760.937695:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972866 is committed 00002000:00010000:1.0:1789760760.938122:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22716416 -> 22720512 00010000:00010000:1.0:1789760760.938140:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55eb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.938152:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55eb5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.938160:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55eb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.938167:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760760.938171:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55eb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.938180:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55eb5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.938201:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55eb5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.938214:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe55eb5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22716416->22720511] (req 22716416->22720511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.938525:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508393088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 72278us (72577us total) trans 4294972866 rc 0/0 00000100:00100000:1.0:1789760760.938535:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5578 00000100:00100000:1.0:1789760760.938545:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739940689746 time 739970940096 state 0x2 ref 0 00000100:00100000:2.0:1789760760.949813:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760760.949958:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508393856 00000100:00100000:1.0:1789760760.950407:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508393856 00000100:00100000:1.0:1789760760.950465:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739940689746 time 739970940096 state 0x0 ref 1 00000100:00100000:1.0:1789760760.950481:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739971200512 time 739970940096 state 0x1 ref 1 00000100:00100000:1.0:1789760760.950512:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739971200512 time 740055222884 state 0x0 ref 1 00000100:00100000:1.0:1789760760.950525:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508393856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760760.950575:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22720512, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:1.0:1789760760.950584:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ebc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.950600:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760760.950605:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760760.950610:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ebc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.950627:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ebc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.995173:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972867 is committed 00002000:00010000:1.0:1789760760.995356:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22720512 -> 22724608 00010000:00010000:1.0:1789760760.995371:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ebc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.995384:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ebc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.995393:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ebc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.995401:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760760.995406:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ebc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.995415:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ebc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.995435:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ebc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760760.995442:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ebc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22720512->22724607] (req 22720512->22724607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760760.995541:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508393856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45014us (45734us total) trans 4294972867 rc 0/0 00000100:00100000:1.0:1789760760.995552:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5579 00000100:00100000:1.0:1789760760.995562:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739971200512 time 740055222884 state 0x2 ref 0 00000100:00100000:3.0:1789760761.005278:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.005454:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508393984 00000100:00100000:1.0:1789760761.005469:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508393984 00000100:00100000:1.0:1789760761.005551:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 739971200512 time 740055222884 state 0x0 ref 1 00000100:00100000:1.0:1789760761.005566:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740055483300 time 740055222884 state 0x1 ref 1 00000100:00100000:1.0:1789760761.005606:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740055483300 time 740110315839 state 0x0 ref 1 00000100:00100000:1.0:1789760761.005620:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508393984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.005668:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22724608, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:1.0:1789760761.005678:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ec3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.005694:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.005698:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.005705:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ec3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.005719:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ec3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760761.006167:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760761.064446:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972868 is committed 00002000:00010000:1.0:1789760761.064957:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22724608 -> 22728704 00010000:00010000:1.0:1789760761.064980:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ec3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.064996:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ec3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.065007:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ec3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.065015:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.065021:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ec3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.065032:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ec3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.065057:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ec3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.065065:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55ec3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22724608->22728703] (req 22724608->22728703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.065238:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508393984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59614us (59970us total) trans 4294972868 rc 0/0 00000100:00100000:1.0:1789760761.065253:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5580 00000100:00100000:1.0:1789760761.065266:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740055483300 time 740110315839 state 0x2 ref 0 00000100:00100000:2.0:1789760761.074169:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.074399:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508394112 00000100:00100000:1.0:1789760761.074413:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508394112 00000100:00100000:1.0:1789760761.074488:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740055483300 time 740110315839 state 0x0 ref 1 00000100:00100000:1.0:1789760761.074622:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740110576255 time 740110315839 state 0x1 ref 1 00000100:00100000:1.0:1789760761.074694:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740110576255 time 740179404170 state 0x0 ref 1 00000100:00100000:1.0:1789760761.074710:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508394112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.074779:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22728704, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:1.0:1789760761.074790:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55eca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.074810:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.074816:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.074823:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55eca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.074840:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55eca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.151675:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972869 is committed 00002000:00010000:3.0:1789760761.151998:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22728704 -> 22732800 00010000:00010000:3.0:1789760761.152024:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55eca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.152039:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55eca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.152050:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55eca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.152059:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760761.152065:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55eca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.152075:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55eca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.152113:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55eca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.152122:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe55eca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22728704->22732799] (req 22728704->22732799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.152252:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508394112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 77540us (78093us total) trans 4294972869 rc 0/0 00000100:00100000:3.0:1789760761.152265:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5581 00000100:00100000:3.0:1789760761.152649:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740110576255 time 740179404170 state 0x2 ref 0 00000100:00100000:2.0:1789760761.156220:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.156360:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508394240 00000100:00100000:1.0:1789760761.156379:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508394240 00000100:00100000:1.0:1789760761.156458:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740110576255 time 740179404170 state 0x0 ref 1 00000100:00100000:1.0:1789760761.156477:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740179664586 time 740179404170 state 0x1 ref 1 00000100:00100000:1.0:1789760761.156505:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740179664586 time 740261215776 state 0x0 ref 1 00000100:00100000:1.0:1789760761.156520:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508394240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.156585:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22732800, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:1.0:1789760761.156597:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55ed1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.156617:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.156622:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.156630:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55ed1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.156646:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55ed1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.204021:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22732800 -> 22736896 00010000:00010000:3.0:1789760761.204050:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55ed1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.204066:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55ed1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.204076:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55ed1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.204084:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760761.204090:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55ed1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.204100:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55ed1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.204125:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55ed1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.204140:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe55ed1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22732800->22736895] (req 22732800->22736895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.204267:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508394240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47742us (48049us total) trans 4294972870 rc 0/0 00000100:00100000:3.0:1789760761.204325:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5582 00000100:00100000:3.0:1789760761.204339:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740179664586 time 740261215776 state 0x2 ref 0 00000001:00080000:0.0:1789760761.205082:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972870 is committed 00000100:00100000:2.0:1789760761.208370:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.208538:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508394368 00000100:00100000:1.0:1789760761.208552:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508394368 00000100:00100000:1.0:1789760761.208625:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740179664586 time 740261215776 state 0x0 ref 1 00000100:00100000:1.0:1789760761.208645:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740261476192 time 740261215776 state 0x1 ref 1 00000100:00100000:1.0:1789760761.208677:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740261476192 time 740313387496 state 0x0 ref 1 00000100:00100000:1.0:1789760761.208694:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508394368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.208757:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22736896, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:1.0:1789760761.208768:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ed8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.208788:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.208793:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.208801:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ed8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.208818:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ed8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.280771:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972871 is committed 00002000:00010000:1.0:1789760761.281165:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22736896 -> 22740992 00010000:00010000:1.0:1789760761.281188:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ed8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.281203:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ed8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.281213:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ed8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.281221:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.281228:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ed8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.281238:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ed8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.282120:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ed8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.282133:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ed8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22736896->22740991] (req 22736896->22740991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.282488:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508394368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 73787us (74126us total) trans 4294972871 rc 0/0 00000100:00100000:1.0:1789760761.282629:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5583 00000100:00100000:1.0:1789760761.282643:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740261476192 time 740313387496 state 0x2 ref 0 00000100:00100000:1.0:1789760761.292873:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760761.293216:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508394496 00000100:00100000:3.0:1789760761.293230:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508394496 00000100:00100000:3.0:1789760761.293359:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740261476192 time 740313387496 state 0x0 ref 1 00000100:00100000:3.0:1789760761.293378:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740313647912 time 740313387496 state 0x1 ref 1 00000100:00100000:3.0:1789760761.293490:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740313647912 time 740398198734 state 0x0 ref 1 00000100:00100000:3.0:1789760761.293507:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508394496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760761.293656:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22740992, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:3.0:1789760761.293667:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55edf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.293688:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760761.293693:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.293701:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55edf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.293722:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55edf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760761.355436:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22740992 -> 22745088 00010000:00010000:1.0:1789760761.355464:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55edf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.355479:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55edf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.355490:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55edf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.355503:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.355508:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55edf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.355518:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55edf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.355542:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55edf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.355549:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe55edf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22740992->22745087] (req 22740992->22745087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.355672:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508394496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 62161us (62806us total) trans 4294972872 rc 0/0 00000100:00100000:1.0:1789760761.355685:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5584 00000100:00100000:1.0:1789760761.355698:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740313647912 time 740398198734 state 0x2 ref 0 00000001:00080000:3.0:1789760761.356457:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972872 is committed 00000100:00100000:2.0:1789760761.363365:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760761.363563:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508394624 00000100:00100000:3.0:1789760761.363575:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508394624 00000100:00100000:3.0:1789760761.363649:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740313647912 time 740398198734 state 0x0 ref 1 00000100:00100000:3.0:1789760761.363667:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740398459150 time 740398198734 state 0x1 ref 1 00000100:00100000:3.0:1789760761.363700:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740398459150 time 740468410488 state 0x0 ref 1 00000100:00100000:3.0:1789760761.363726:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508394624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760761.363794:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22745088, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:3.0:1789760761.363805:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ee6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.363824:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760761.363830:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.363838:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ee6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.363855:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ee6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760761.416037:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22745088 -> 22749184 00010000:00010000:1.0:1789760761.416066:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ee6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.416081:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ee6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.416098:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ee6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.416106:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.416111:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ee6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.416121:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ee6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.416147:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ee6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.416157:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe55ee6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22745088->22749183] (req 22745088->22749183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.416321:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508394624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52549us (52922us total) trans 4294972873 rc 0/0 00000100:00100000:1.0:1789760761.416334:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5585 00000100:00100000:1.0:1789760761.416347:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740398459150 time 740468410488 state 0x2 ref 0 00000001:00080000:0.0:1789760761.416506:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972873 is committed 00000100:00100000:2.0:1789760761.419575:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.419680:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508394752 00000100:00100000:1.0:1789760761.419693:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508394752 00000100:00100000:1.0:1789760761.419771:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740398459150 time 740468410488 state 0x0 ref 1 00000100:00100000:1.0:1789760761.419790:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740468670904 time 740468410488 state 0x1 ref 1 00000100:00100000:1.0:1789760761.419824:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740468670904 time 740524534637 state 0x0 ref 1 00000100:00100000:1.0:1789760761.419837:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508394752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.419903:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22749184, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:1.0:1789760761.419914:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55eed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.419944:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.419949:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.419967:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55eed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.419986:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55eed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.487434:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972874 is committed 00002000:00010000:1.0:1789760761.487615:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22749184 -> 22753280 00010000:00010000:1.0:1789760761.487636:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55eed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.487651:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55eed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.487660:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55eed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.487668:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.487676:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55eed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.487687:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55eed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.487713:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55eed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.487721:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55eed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22749184->22753279] (req 22749184->22753279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.487864:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508394752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 68021us (68297us total) trans 4294972874 rc 0/0 00000100:00100000:1.0:1789760761.487876:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5586 00000100:00100000:1.0:1789760761.487888:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740468670904 time 740524534637 state 0x2 ref 0 00000100:00100000:3.0:1789760761.494578:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.494759:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508394880 00000100:00100000:1.0:1789760761.494773:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508394880 00000100:00100000:1.0:1789760761.494854:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740468670904 time 740524534637 state 0x0 ref 1 00000100:00100000:1.0:1789760761.494873:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740524795053 time 740524534637 state 0x1 ref 1 00000100:00100000:1.0:1789760761.494908:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740524795053 time 740599618251 state 0x0 ref 1 00000100:00100000:1.0:1789760761.494921:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508394880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.495001:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22753280, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0xb007 00010000:00010000:1.0:1789760761.495012:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ef4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.495033:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.495039:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.495047:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ef4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.495064:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ef4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.533893:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972875 is committed 00002000:00010000:1.0:1789760761.534132:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22753280 -> 22757376 00002000:00010000:1.0:1789760761.534137:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760761 -> 1789760762 00002000:00010000:1.0:1789760761.534140:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760761 -> 1789760762 00010000:00010000:1.0:1789760761.534159:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ef4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.534176:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ef4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.534186:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ef4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.534195:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.534201:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ef4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.534217:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ef4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.534244:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ef4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.534253:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55ef4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22753280->22757375] (req 22753280->22757375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.534622:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508394880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39695us (40053us total) trans 4294972875 rc 0/0 00000100:00100000:1.0:1789760761.534636:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5587 00000100:00100000:1.0:1789760761.534649:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740524795053 time 740599618251 state 0x2 ref 0 00000100:00100000:3.0:1789760761.537907:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.538720:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508395008 00000100:00100000:1.0:1789760761.538738:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508395008 00000100:00100000:0.0:1789760761.538982:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740524795053 time 740599618251 state 0x0 ref 1 00000100:00100000:0.0:1789760761.539002:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740599878667 time 740599618251 state 0x1 ref 1 00000100:00100000:0.0:1789760761.539163:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740599878667 time 740643871629 state 0x0 ref 1 00000100:00100000:0.0:1789760761.539182:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508395008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760761.539250:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22757376, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:0.0:1789760761.539262:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55efb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.539321:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760761.539327:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760761.539335:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55efb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760761.539354:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55efb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760761.581368:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22757376 -> 22761472 00010000:00010000:1.0:1789760761.581394:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55efb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.581566:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55efb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.581576:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55efb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.581589:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.581595:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55efb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.581605:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55efb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.581630:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55efb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.581638:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe55efb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22757376->22761471] (req 22757376->22761471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.581765:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508395008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42581us (43869us total) trans 4294972876 rc 0/0 00000100:00100000:1.0:1789760761.581779:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5588 00000100:00100000:1.0:1789760761.581791:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740599878667 time 740643871629 state 0x2 ref 0 00000001:00080000:2.0:1789760761.581957:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972876 is committed 00000100:00100000:2.0:1789760761.587070:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.587163:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508395136 00000100:00100000:1.0:1789760761.587176:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508395136 00000100:00100000:1.0:1789760761.587254:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740599878667 time 740643871629 state 0x0 ref 1 00000100:00100000:1.0:1789760761.587282:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740644132045 time 740643871629 state 0x1 ref 1 00000100:00100000:1.0:1789760761.587369:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740644132045 time 740692079693 state 0x0 ref 1 00000100:00100000:1.0:1789760761.587387:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508395136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.587614:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22761472, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760761.587626:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f02 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.587647:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.587652:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.587661:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.587679:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.599730:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760761.637918:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972877 is committed 00002000:00010000:1.0:1789760761.638163:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22761472 -> 22765568 00010000:00010000:1.0:1789760761.638184:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.638200:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.638210:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.638219:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.638225:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.638235:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.638261:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.638270:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22761472->22765567] (req 22761472->22765567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.638637:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508395136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51245us (51575us total) trans 4294972877 rc 0/0 00000100:00100000:1.0:1789760761.638649:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5589 00000100:00100000:1.0:1789760761.638662:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740644132045 time 740692079693 state 0x2 ref 0 00000100:00100000:3.0:1789760761.644724:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.644834:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508395264 00000100:00100000:1.0:1789760761.644847:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508395264 00000100:00100000:1.0:1789760761.644919:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740644132045 time 740692079693 state 0x0 ref 1 00000100:00100000:1.0:1789760761.644951:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740692340109 time 740692079693 state 0x1 ref 1 00000100:00100000:1.0:1789760761.645108:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740692340109 time 740749816338 state 0x0 ref 1 00000100:00100000:1.0:1789760761.645126:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508395264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.645198:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22765568, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760761.645210:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.645231:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.645237:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.645245:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.645262:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760761.683975:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22765568 -> 22769664 00010000:00010000:1.0:1789760761.684003:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.684019:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.684030:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.684037:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.684043:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.684053:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.684078:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.684087:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22765568->22769663] (req 22765568->22769663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.684222:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508395264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39091us (39505us total) trans 4294972878 rc 0/0 00000100:00100000:1.0:1789760761.684234:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5590 00000100:00100000:1.0:1789760761.684246:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740692340109 time 740749816338 state 0x2 ref 0 00000001:00080000:2.0:1789760761.684467:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972878 is committed 00000100:00100000:3.0:1789760761.696937:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.698024:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508395392 00000100:00100000:1.0:1789760761.698044:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508395392 00000100:00100000:1.0:1789760761.698127:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740692340109 time 740749816338 state 0x0 ref 1 00000100:00100000:1.0:1789760761.698147:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740750076754 time 740749816338 state 0x1 ref 1 00000100:00100000:1.0:1789760761.698189:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740750076754 time 740802899882 state 0x0 ref 1 00000100:00100000:1.0:1789760761.698204:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508395392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.698270:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22769664, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760761.698280:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.698346:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.698351:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.698359:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.698381:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.727101:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760761.727134:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534077568:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760761.727229:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760761.727381:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760761.727392:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b644e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534077696:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760761.727423:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760761.727465:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534077568 00000100:00100000:1.0:1789760761.727476:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534077568 00000100:00100000:1.0:1789760761.727544:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534077696 00000100:00100000:1.0:1789760761.727548:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534077696 00000100:00100000:1.0:1789760761.727571:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:1.0:1789760761.727581:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534077568:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760761.727613:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3835691008 avail 3821010944 00000100:00100000:1.0:1789760761.727664:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534077568:12345-0@lo:13:osp-pre-0-0.0 Request processed in 83us (435us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760761.727672:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:1.0:1789760761.727693:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:1.0:1789760761.727699:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534077696:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760761.727726:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760761.727760:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534077696:12345-0@lo:13:osp-pre-1-1.0 Request processed in 60us (336us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760761.727766:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00080000:1.0:1789760761.727784:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760761.727788:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760761.727828:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b644e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534077696:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760761.727867:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048846d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534077568:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760761.728893:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760761.728911:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0488649d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534077824:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760761.728978:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.729020:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534077824 00000100:00100000:0.0:1789760761.729028:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534077824 00000100:00100000:0.0:1789760761.729079:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:0.0:1789760761.729087:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534077824:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760761.729110:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527630336 pending 1703936 free 3835691008 avail 3821010944 00000100:00100000:0.0:1789760761.729148:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534077824:12345-0@lo:13:osp-pre-0-1.0 Request processed in 61us (170us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760761.729155:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00080000:0.0:1789760761.729174:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760761.729206:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0488649d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534077824:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760761.729595:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3473:1876699534078080:0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.729634:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.729668:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534078592:0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.729695:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.729713:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c9c7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3473:1876699534079104:0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.729738:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.729767:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:0.0:1789760761.729773:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b644e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534077952:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760761.729797:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.729814:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b644000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3473:1876699534078464:0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.729834:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.729851:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c9c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3473:1876699534078976:0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.729873:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.729903:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534078592 00000100:00100000:0.0:1789760761.729909:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534078592 00000100:00100000:0.0:1789760761.729959:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 446 00000100:00100000:0.0:1789760761.729967:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534078592:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.730012:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534078592:12345-0@lo:400:kworker.0 Request processed in 46us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760761.730019:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 446 00000100:00100000:0.0:1789760761.730043:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534077952 00000100:00100000:0.0:1789760761.730047:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534077952 00000100:00100000:0.0:1789760761.730076:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:0.0:1789760761.730082:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534077952:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760761.730096:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760761.730126:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534077952:12345-0@lo:13:osp-pre-1-0.0 Request processed in 44us (327us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760761.730132:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00000100:00080000:0.0:1789760761.730145:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760761.730154:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b645500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534078208:0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.730180:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.730199:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534078720:0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.730221:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.730239:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c9c6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534079232:0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.730263:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.730391:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1789760761.730398:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534078336:0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.730425:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.730442:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c9c7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534078848:0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.730463:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760761.730489:0:8103:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534079232 00000100:00100000:0.0:1789760761.730493:0:8103:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534079232 00000100:00100000:0.0:1789760761.730523:0:8103:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 234 00000100:00100000:0.0:1789760761.730530:0:8103:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534079232:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.730565:0:8103:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534079232:12345-0@lo:400:kworker.0 Request processed in 36us (301us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760761.730571:0:8103:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 234 00000100:00100000:0.0:1789760761.730589:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534078336 00000100:00100000:0.0:1789760761.730593:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534078336 00000100:00100000:0.0:1789760761.730615:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 447 00000100:00100000:0.0:1789760761.730621:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534078336:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.730650:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534078336:12345-0@lo:400:kworker.0 Request processed in 30us (225us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760761.730656:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 447 00000100:00100000:0.0:1789760761.730711:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b644e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534077952:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760761.730725:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534078592:0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.730766:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534078336:0@lo:400:kworker.0 00000100:00100000:0.0:1789760761.730776:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c9c6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534079232:0@lo:400:kworker.0 00000100:00100000:1.0:1789760761.730906:0:6460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534078080 00000100:00100000:1.0:1789760761.730911:0:6460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534078080 00000100:00100000:1.0:1789760761.730956:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 480 00000100:00100000:1.0:1789760761.730963:0:6460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cac450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3473:x1876699534078080:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760761.731008:0:6460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cac450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3473:x1876699534078080:12345-0@lo:400:kworker.0 Request processed in 46us (1374us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760761.731014:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 480 00000100:00100000:2.0:1789760761.731086:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3473:1876699534078080:0@lo:400:kworker.0 00000100:00100000:3.0:1789760761.731421:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534079104 00000100:00100000:3.0:1789760761.731427:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534079104 00000100:00100000:3.0:1789760761.731457:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534078464 00000100:00100000:3.0:1789760761.731460:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534078464 00000100:00100000:3.0:1789760761.731472:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534078976 00000100:00100000:3.0:1789760761.731474:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534078976 00000100:00100000:3.0:1789760761.731484:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534078208 00000100:00100000:3.0:1789760761.731486:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534078208 00000100:00100000:3.0:1789760761.731495:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534078720 00000100:00100000:3.0:1789760761.731497:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534078720 00000100:00100000:3.0:1789760761.731506:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534078848 00000100:00100000:3.0:1789760761.731508:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534078848 00000100:00100000:3.0:1789760761.731520:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 933 00000100:00100000:3.0:1789760761.731527:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3473:x1876699534079104:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760761.731613:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3473:x1876699534079104:12345-0@lo:400:kworker.0 Request processed in 87us (1873us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760761.731620:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 933 00000100:00100000:3.0:1789760761.731632:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 934 00000100:00100000:3.0:1789760761.731637:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3473:x1876699534078464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760761.731662:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3473:x1876699534078464:12345-0@lo:400:kworker.0 Request processed in 26us (1827us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760761.731668:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 934 00000100:00100000:3.0:1789760761.731677:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 935 00000100:00100000:3.0:1789760761.731681:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3473:x1876699534078976:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760761.731704:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3473:x1876699534078976:12345-0@lo:400:kworker.0 Request processed in 22us (1830us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760761.731709:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 935 00000100:00100000:3.0:1789760761.731718:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 936 00000100:00100000:3.0:1789760761.731723:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534078208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760761.731757:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534078208:12345-0@lo:400:kworker.0 Request processed in 34us (1577us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760761.731763:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 936 00000100:00100000:3.0:1789760761.731772:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 937 00000100:00100000:3.0:1789760761.731777:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534078720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760761.731798:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534078720:12345-0@lo:400:kworker.0 Request processed in 21us (1577us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760761.731804:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 937 00000100:00100000:3.0:1789760761.731811:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 938 00000100:00100000:3.0:1789760761.731816:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534078848:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760761.731835:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534078848:12345-0@lo:400:kworker.0 Request processed in 19us (1371us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760761.731840:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 938 00000100:00080000:0.0:1789760761.734607:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760761.734620:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789760761.735043:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c9c7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534078848:0@lo:400:kworker.0 00000100:00100000:1.0:1789760761.735064:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b645500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534078208:0@lo:400:kworker.0 00000100:00100000:1.0:1789760761.735078:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048846d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534078720:0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.735200:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c9c7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3473:1876699534079104:0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.735231:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b644000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3473:1876699534078464:0@lo:400:kworker.0 00000100:00100000:2.0:1789760761.735243:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c9c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3473:1876699534078976:0@lo:400:kworker.0 00000001:00080000:2.0:1789760761.753408:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972879 is committed 00002000:00010000:1.0:1789760761.753575:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22769664 -> 22773760 00010000:00010000:1.0:1789760761.753594:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.753609:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.753620:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.753629:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.753635:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.753646:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.753669:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.753678:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22769664->22773759] (req 22769664->22773759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.753848:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508395392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55630us (56918us total) trans 4294972879 rc 0/0 00000100:00100000:1.0:1789760761.753862:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5591 00000100:00100000:1.0:1789760761.753876:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740750076754 time 740802899882 state 0x2 ref 0 00000100:00100000:3.0:1789760761.757767:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.757842:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508395520 00000100:00100000:1.0:1789760761.757857:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508395520 00000100:00100000:1.0:1789760761.757939:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740750076754 time 740802899882 state 0x0 ref 1 00000100:00100000:1.0:1789760761.757962:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740803160298 time 740802899882 state 0x1 ref 1 00000100:00100000:1.0:1789760761.758002:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740803160298 time 740862712175 state 0x0 ref 1 00000100:00100000:1.0:1789760761.758015:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508395520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.758081:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22773760, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760761.758093:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.758113:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.758118:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.758126:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.758143:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760761.791129:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972880 is committed 00002000:00010000:1.0:1789760761.791249:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22773760 -> 22777856 00010000:00010000:1.0:1789760761.791387:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.791403:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.791414:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.791422:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.791427:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.791437:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.791466:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.791474:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22773760->22777855] (req 22773760->22777855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.791620:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508395520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33601us (33860us total) trans 4294972880 rc 0/0 00000100:00100000:1.0:1789760761.791632:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5592 00000100:00100000:1.0:1789760761.791643:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740803160298 time 740862712175 state 0x2 ref 0 00000100:00100000:2.0:1789760761.795918:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.796096:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508395648 00000100:00100000:1.0:1789760761.796108:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508395648 00000100:00100000:1.0:1789760761.796184:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740803160298 time 740862712175 state 0x0 ref 1 00000100:00100000:1.0:1789760761.796206:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740862972591 time 740862712175 state 0x1 ref 1 00000100:00100000:1.0:1789760761.796326:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740862972591 time 740900989116 state 0x0 ref 1 00000100:00100000:1.0:1789760761.796341:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508395648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.796411:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22777856, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760761.796422:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.796442:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.796448:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.796455:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.796472:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760761.831822:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22777856 -> 22781952 00010000:00010000:1.0:1789760761.831850:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.831866:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.831876:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.831885:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.831891:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.831902:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.831939:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.831948:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22777856->22781951] (req 22777856->22781951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.832072:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508395648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35725us (36160us total) trans 4294972881 rc 0/0 00000100:00100000:1.0:1789760761.832085:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5593 00000100:00100000:1.0:1789760761.832098:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740862972591 time 740900989116 state 0x2 ref 0 00000001:00080000:0.0:1789760761.832193:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972881 is committed 00000100:00100000:3.0:1789760761.842474:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.842756:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508395776 00000100:00100000:1.0:1789760761.842769:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508395776 00000100:00100000:1.0:1789760761.842957:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740862972591 time 740900989116 state 0x0 ref 1 00000100:00100000:1.0:1789760761.842978:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740901249532 time 740900989116 state 0x1 ref 1 00000100:00100000:1.0:1789760761.843064:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740901249532 time 740947772502 state 0x0 ref 1 00000100:00100000:1.0:1789760761.843080:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508395776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.843147:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22781952, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760761.843158:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.843180:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760761.843185:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.843193:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.843210:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760761.899158:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972882 is committed 00002000:00010000:1.0:1789760761.899408:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22781952 -> 22786048 00010000:00010000:1.0:1789760761.899428:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.899443:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.899454:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.899462:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760761.899469:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.899478:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.899511:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.899520:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22781952->22786047] (req 22781952->22786047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760761.899967:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508395776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56882us (57795us total) trans 4294972882 rc 0/0 00000100:00100000:1.0:1789760761.899983:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5594 00000100:00100000:1.0:1789760761.899996:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740901249532 time 740947772502 state 0x2 ref 0 00000100:00100000:3.0:1789760761.902884:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760761.903064:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508395904 00000100:00100000:1.0:1789760761.903077:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508395904 00000100:00100000:1.0:1789760761.903144:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740901249532 time 740947772502 state 0x0 ref 1 00000100:00100000:1.0:1789760761.903162:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740948032918 time 740947772502 state 0x1 ref 1 00000100:00100000:1.0:1789760761.903193:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740948032918 time 741007903698 state 0x0 ref 1 00000100:00100000:1.0:1789760761.903206:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508395904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760761.903280:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22786048, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760761.903320:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.903341:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760761.903346:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760761.903352:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760761.903369:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.059725:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972883 is committed 00002000:00010000:1.0:1789760762.059751:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22786048 -> 22790144 00010000:00010000:1.0:1789760762.059772:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.059789:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.059800:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.059809:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760762.059815:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.059825:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.059851:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.059861:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22786048->22790143] (req 22786048->22790143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.060036:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508395904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 156824us (157158us total) trans 4294972883 rc 0/0 00000100:00100000:1.0:1789760762.060050:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5595 00000100:00100000:1.0:1789760762.060062:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740948032918 time 741007903698 state 0x2 ref 0 00000100:00100000:2.0:1789760762.063586:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760762.063634:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508396032 00000100:00100000:1.0:1789760762.063648:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508396032 00000100:00100000:1.0:1789760762.063720:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 740948032918 time 741007903698 state 0x0 ref 1 00000100:00100000:1.0:1789760762.063740:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741008164114 time 741007903698 state 0x1 ref 1 00000100:00100000:1.0:1789760762.063772:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741008164114 time 741168482102 state 0x0 ref 1 00000100:00100000:1.0:1789760762.063784:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508396032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760762.063856:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22790144, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760762.063866:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.063885:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760762.063891:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.063899:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.063915:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760762.115915:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22790144 -> 22794240 00010000:00010000:1.0:1789760762.115956:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.115972:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.115988:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.115996:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760762.116002:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.116013:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.116038:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.116047:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22790144->22794239] (req 22790144->22794239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.116176:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508396032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52386us (52604us total) trans 4294972884 rc 0/0 00000100:00100000:1.0:1789760762.116188:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5596 00000100:00100000:1.0:1789760762.116200:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741008164114 time 741168482102 state 0x2 ref 0 00000001:00080000:2.0:1789760762.116435:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972884 is committed 00000100:00100000:3.0:1789760762.120143:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760762.120254:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508396160 00000100:00100000:1.0:1789760762.122552:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508396160 00000100:00100000:1.0:1789760762.122653:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741008164114 time 741168482102 state 0x0 ref 1 00000100:00100000:1.0:1789760762.122678:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741168742518 time 741168482102 state 0x1 ref 1 00000100:00100000:1.0:1789760762.122853:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741168742518 time 741227562526 state 0x0 ref 1 00000100:00100000:1.0:1789760762.122869:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508396160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760762.122957:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22794240, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760762.122968:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.122988:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760762.122993:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.123001:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.123018:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760762.177545:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22794240 -> 22798336 00010000:00010000:1.0:1789760762.177580:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.177592:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.177599:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.177605:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760762.177609:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.177618:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.177638:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.177642:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22794240->22798335] (req 22794240->22798335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.177764:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508396160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54891us (57629us total) trans 4294972885 rc 0/0 00000100:00100000:1.0:1789760762.177778:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5597 00000100:00100000:1.0:1789760762.177791:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741168742518 time 741227562526 state 0x2 ref 0 00000001:00080000:3.0:1789760762.177982:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972885 is committed 00000100:00100000:0.0:1789760762.180159:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760762.180357:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508396288 00000100:00100000:1.0:1789760762.180368:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508396288 00000100:00100000:1.0:1789760762.180444:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741168742518 time 741227562526 state 0x0 ref 1 00000100:00100000:1.0:1789760762.180464:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741227822942 time 741227562526 state 0x1 ref 1 00000100:00100000:1.0:1789760762.180501:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741227822942 time 741285211091 state 0x0 ref 1 00000100:00100000:1.0:1789760762.180524:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508396288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760762.180587:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22798336, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760762.180598:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.180619:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760762.180624:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.180816:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.180838:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.223510:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972886 is committed 00002000:00010000:1.0:1789760762.223818:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22798336 -> 22802432 00010000:00010000:1.0:1789760762.223853:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.223870:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.223879:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.223887:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760762.223893:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.223904:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.224063:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.224072:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22798336->22802431] (req 22798336->22802431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.224203:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508396288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43682us (44046us total) trans 4294972886 rc 0/0 00000100:00100000:1.0:1789760762.224215:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5598 00000100:00100000:1.0:1789760762.224227:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741227822942 time 741285211091 state 0x2 ref 0 00000100:00100000:0.0:1789760762.230911:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760762.231020:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508396416 00000100:00100000:1.0:1789760762.231033:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508396416 00000100:00100000:1.0:1789760762.231109:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741227822942 time 741285211091 state 0x0 ref 1 00000100:00100000:1.0:1789760762.231127:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741285471507 time 741285211091 state 0x1 ref 1 00000100:00100000:1.0:1789760762.231157:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741285471507 time 741335867085 state 0x0 ref 1 00000100:00100000:1.0:1789760762.231168:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508396416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760762.231231:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22802432, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760762.231241:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.231260:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760762.231266:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.231281:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.231333:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.277502:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972887 is committed 00002000:00010000:1.0:1789760762.277709:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22802432 -> 22806528 00010000:00010000:1.0:1789760762.277730:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.277747:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.277758:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.277766:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760762.277772:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.277783:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.277808:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.277818:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22802432->22806527] (req 22802432->22806527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760762.277993:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508396416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46818us (47088us total) trans 4294972887 rc 0/0 00000100:00100000:1.0:1789760762.278007:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5599 00000100:00100000:1.0:1789760762.278020:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741285471507 time 741335867085 state 0x2 ref 0 00000100:00100000:3.0:1789760762.281995:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760762.282096:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508396544 00000100:00100000:1.0:1789760762.282109:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508396544 00000100:00100000:1.0:1789760762.282182:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741285471507 time 741335867085 state 0x0 ref 1 00000100:00100000:1.0:1789760762.282202:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741336127501 time 741335867085 state 0x1 ref 1 00000100:00100000:1.0:1789760762.282235:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741336127501 time 741386945445 state 0x0 ref 1 00000100:00100000:1.0:1789760762.282248:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508396544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760762.282540:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22806528, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760762.282553:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.282574:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760762.282580:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.282588:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.282606:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.326166:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972888 is committed 00002000:00010000:0.0:1789760762.326356:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22806528 -> 22810624 00010000:00010000:0.0:1789760762.326376:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.326392:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.326403:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.326411:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760762.326417:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.326427:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.326455:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.326471:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe55f4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22806528->22810623] (req 22806528->22810623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.326588:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508396544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44333us (44600us total) trans 4294972888 rc 0/0 00000100:00100000:0.0:1789760762.326600:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5600 00000100:00100000:0.0:1789760762.326613:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741336127501 time 741386945445 state 0x2 ref 0 00000100:00100000:0.0:1789760762.330933:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760762.331035:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508396672 00000100:00100000:1.0:1789760762.331048:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508396672 00000100:00100000:1.0:1789760762.331126:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741336127501 time 741386945445 state 0x0 ref 1 00000100:00100000:1.0:1789760762.331146:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741387205861 time 741386945445 state 0x1 ref 1 00000100:00100000:1.0:1789760762.331182:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741387205861 time 741435892365 state 0x0 ref 1 00000100:00100000:1.0:1789760762.331196:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508396672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760762.331278:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22810624, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760762.331332:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.331355:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760762.331360:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.331369:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.331388:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.378737:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22810624 -> 22814720 00010000:00010000:3.0:1789760762.378767:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.378796:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.378806:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.378815:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760762.378820:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.378840:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.378866:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.378873:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe55f56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22810624->22814719] (req 22810624->22814719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.379001:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508396672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47799us (48084us total) trans 4294972889 rc 0/0 00000100:00100000:3.0:1789760762.379018:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5601 00000100:00100000:3.0:1789760762.379030:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741387205861 time 741435892365 state 0x2 ref 0 00000001:00080000:3.0:1789760762.379083:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972889 is committed 00000100:00100000:3.0:1789760762.383882:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760762.384004:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508396800 00000100:00100000:1.0:1789760762.384019:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508396800 00000100:00100000:1.0:1789760762.384097:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741387205861 time 741435892365 state 0x0 ref 1 00000100:00100000:1.0:1789760762.384118:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741436152781 time 741435892365 state 0x1 ref 1 00000100:00100000:1.0:1789760762.384154:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741436152781 time 741488863906 state 0x0 ref 1 00000100:00100000:1.0:1789760762.384169:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508396800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760762.384236:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22814720, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:1.0:1789760762.384248:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55f5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.384277:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:1.0:1789760762.384321:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760762.384330:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55f5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.384348:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55f5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.437636:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972890 is committed 00002000:00010000:2.0:1789760762.437763:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22814720 -> 22818816 00010000:00010000:2.0:1789760762.437789:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55f5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.437804:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55f5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.437820:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55f5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.437828:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760762.437834:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55f5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.437843:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55f5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.437869:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55f5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.437877:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe55f5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22814720->22818815] (req 22814720->22818815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.438134:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508396800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53959us (54258us total) trans 4294972890 rc 0/0 00000100:00100000:2.0:1789760762.438147:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5602 00000100:00100000:2.0:1789760762.438159:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741436152781 time 741488863906 state 0x2 ref 0 00000100:00100000:2.0:1789760762.442613:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760762.442704:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508396928 00000100:00100000:0.0:1789760762.442718:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508396928 00000100:00100000:0.0:1789760762.442801:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741436152781 time 741488863906 state 0x0 ref 1 00000100:00100000:0.0:1789760762.442820:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741489124322 time 741488863906 state 0x1 ref 1 00000100:00100000:0.0:1789760762.442864:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741489124322 time 741547565240 state 0x0 ref 1 00000100:00100000:0.0:1789760762.442878:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508396928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760762.442953:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22818816, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:0.0:1789760762.442964:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55f64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.442985:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760762.442990:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760762.442998:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55f64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.443015:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55f64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.483076:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22818816 -> 22822912 00010000:00010000:3.0:1789760762.483102:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55f64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.483117:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55f64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.483127:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55f64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.483134:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760762.483139:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55f64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.483149:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55f64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.483176:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55f64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.483185:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe55f64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22818816->22822911] (req 22818816->22822911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.483355:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508396928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40472us (40755us total) trans 4294972891 rc 0/0 00000100:00100000:3.0:1789760762.483367:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5603 00000100:00100000:3.0:1789760762.483380:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741489124322 time 741547565240 state 0x2 ref 0 00000100:00100000:3.0:1789760762.485976:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760762.486035:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508397056 00000100:00100000:3.0:1789760762.486047:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508397056 00000100:00100000:3.0:1789760762.486122:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741489124322 time 741547565240 state 0x0 ref 1 00000100:00100000:3.0:1789760762.486140:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741547825656 time 741547565240 state 0x1 ref 1 00000100:00100000:3.0:1789760762.486170:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741547825656 time 741590880133 state 0x0 ref 1 00000100:00100000:3.0:1789760762.486183:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508397056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760762.486247:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22822912, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0xb007 00010000:00010000:3.0:1789760762.486257:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55f6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.486307:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760762.486312:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.486319:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55f6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.486336:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55f6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.487636:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972891 is committed 00000001:00080000:0.0:1789760762.541346:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972892 is committed 00002000:00010000:3.0:1789760762.541486:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22822912 -> 22827008 00002000:00010000:3.0:1789760762.541492:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760762 -> 1789760763 00002000:00010000:3.0:1789760762.541494:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760762 -> 1789760763 00010000:00010000:3.0:1789760762.541512:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55f6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.541527:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55f6b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.541536:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55f6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.541544:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760762.541551:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55f6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.541563:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55f6b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.541589:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55f6b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.541598:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55f6b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22822912->22827007] (req 22822912->22827007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.541979:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508397056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55790us (56010us total) trans 4294972892 rc 0/0 00000100:00100000:3.0:1789760762.541992:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5604 00000100:00100000:3.0:1789760762.542004:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741547825656 time 741590880133 state 0x2 ref 0 00000100:00100000:3.0:1789760762.545681:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760762.545979:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508397184 00000100:00100000:2.0:1789760762.545992:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508397184 00000100:00100000:2.0:1789760762.546067:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741547825656 time 741590880133 state 0x0 ref 1 00000100:00100000:2.0:1789760762.546092:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741591140549 time 741590880133 state 0x1 ref 1 00000100:00100000:2.0:1789760762.546186:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741591140549 time 741650894716 state 0x0 ref 1 00000100:00100000:2.0:1789760762.546202:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508397184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760762.546275:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22827008, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:2.0:1789760762.546327:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.546347:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.546353:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.546361:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.546379:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760762.585099:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22827008 -> 22831104 00010000:00010000:2.0:1789760762.585129:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.585145:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f72 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.585156:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.585170:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760762.585177:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.585187:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f72 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.585212:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f72 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.585222:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f72 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22827008->22831103] (req 22827008->22831103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760762.585375:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972893 is committed 00000100:00100000:2.0:1789760762.585583:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508397184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39377us (39910us total) trans 4294972893 rc 0/0 00000100:00100000:2.0:1789760762.585596:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5605 00000100:00100000:2.0:1789760762.585609:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741591140549 time 741650894716 state 0x2 ref 0 00000100:00100000:2.0:1789760762.590715:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760762.590980:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508397312 00000100:00100000:3.0:1789760762.590994:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508397312 00000100:00100000:3.0:1789760762.591070:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741591140549 time 741650894716 state 0x0 ref 1 00000100:00100000:3.0:1789760762.591091:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741651155132 time 741650894716 state 0x1 ref 1 00000100:00100000:3.0:1789760762.591131:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741651155132 time 741695840142 state 0x0 ref 1 00000100:00100000:3.0:1789760762.591148:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508397312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760762.591214:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22831104, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:3.0:1789760762.591225:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.591245:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760762.591250:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.591258:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.591323:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.622691:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760762.640157:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972894 is committed 00002000:00010000:3.0:1789760762.640610:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22831104 -> 22835200 00010000:00010000:3.0:1789760762.640626:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.640638:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.640647:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.640655:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760762.640659:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.640667:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.640686:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.640694:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22831104->22835199] (req 22831104->22835199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.640892:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508397312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49742us (50196us total) trans 4294972894 rc 0/0 00000100:00100000:3.0:1789760762.640903:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5606 00000100:00100000:3.0:1789760762.640911:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741651155132 time 741695840142 state 0x2 ref 0 00000100:00100000:3.0:1789760762.647197:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760762.649808:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508397440 00000100:00100000:0.0:1789760762.649823:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508397440 00000100:00100000:2.0:1789760762.650018:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741651155132 time 741695840142 state 0x0 ref 1 00000100:00100000:2.0:1789760762.650033:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741696100558 time 741695840142 state 0x1 ref 1 00000100:00100000:2.0:1789760762.650071:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741696100558 time 741754780958 state 0x0 ref 1 00000100:00100000:2.0:1789760762.650087:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508397440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760762.650137:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22835200, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:2.0:1789760762.650146:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.650160:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.650164:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.650170:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.650183:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.692934:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972895 is committed 00002000:00010000:2.0:1789760762.693127:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22835200 -> 22839296 00010000:00010000:2.0:1789760762.693141:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.693154:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.693164:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.693173:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760762.693177:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.693188:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.693208:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.693216:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22835200->22839295] (req 22835200->22839295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.694099:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508397440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44006us (46904us total) trans 4294972895 rc 0/0 00000100:00100000:2.0:1789760762.694111:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5607 00000100:00100000:2.0:1789760762.694122:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741696100558 time 741754780958 state 0x2 ref 0 00000100:00100000:0.0:1789760762.698821:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760762.703160:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508397568 00000100:00100000:2.0:1789760762.703175:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508397568 00000100:00100000:2.0:1789760762.703241:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741696100558 time 741754780958 state 0x0 ref 1 00000100:00100000:2.0:1789760762.703257:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741755041374 time 741754780958 state 0x1 ref 1 00000100:00100000:2.0:1789760762.703279:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741755041374 time 741807989930 state 0x0 ref 1 00000100:00100000:2.0:1789760762.703317:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508397568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760762.703364:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22839296, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:2.0:1789760762.703373:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f87 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.703389:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.703394:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.703400:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.703414:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.750974:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789760762.751005:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b646680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534079360:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760762.751103:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760762.752774:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534079360 00000100:00100000:0.0:1789760762.752792:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534079360 00000100:00100000:0.0:1789760762.752884:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:0.0:1789760762.752897:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534079360:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760762.752969:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789760762.753067:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534079360:12345-0@lo:41:osp-pre-0-1.0 Request processed in 172us (1963us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760762.753075:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:3.0:1789760762.753660:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b646680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534079360:0@lo:41:osp-pre-0-1.0 00002000:00010000:0.0:1789760762.835686:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22839296 -> 22843392 00010000:00010000:0.0:1789760762.835714:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.835730:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.835747:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.835755:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760762.835760:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.835770:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.835797:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760762.835806:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe55f87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22839296->22843391] (req 22839296->22843391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760762.836004:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508397568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 132684us (137187us total) trans 4294972896 rc 0/0 00000100:00100000:0.0:1789760762.836017:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5608 00000100:00100000:0.0:1789760762.836029:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741755041374 time 741807989930 state 0x2 ref 0 00000001:00080000:3.0:1789760762.837145:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972896 is committed 00000100:00100000:1.0:1789760762.840007:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760762.840112:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508397696 00000100:00100000:3.0:1789760762.840126:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508397696 00000100:00100000:3.0:1789760762.840200:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741755041374 time 741807989930 state 0x0 ref 1 00000100:00100000:3.0:1789760762.840221:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741808250346 time 741807989930 state 0x1 ref 1 00000100:00100000:3.0:1789760762.840328:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741808250346 time 741945037430 state 0x0 ref 1 00000100:00100000:3.0:1789760762.840344:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508397696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760762.840416:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22843392, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:3.0:1789760762.840428:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.840447:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760762.840453:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.840460:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.840479:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.879261:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22843392 -> 22847488 00010000:00010000:3.0:1789760762.879321:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.879338:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.879348:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.879355:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760762.879361:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.879370:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.879394:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.879403:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22843392->22847487] (req 22843392->22847487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.879518:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508397696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39172us (39519us total) trans 4294972897 rc 0/0 00000100:00100000:3.0:1789760762.879535:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5609 00000100:00100000:3.0:1789760762.879550:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741808250346 time 741945037430 state 0x2 ref 0 00000001:00080000:2.0:1789760762.879642:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972897 is committed 00000100:00100000:1.0:1789760762.882644:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760762.882751:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508397824 00000100:00100000:2.0:1789760762.882765:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508397824 00000100:00100000:2.0:1789760762.882854:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741808250346 time 741945037430 state 0x0 ref 1 00000100:00100000:2.0:1789760762.882873:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741945297846 time 741945037430 state 0x1 ref 1 00000100:00100000:2.0:1789760762.882905:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741945297846 time 741987614641 state 0x0 ref 1 00000100:00100000:2.0:1789760762.882928:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508397824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760762.883000:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22847488, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:2.0:1789760762.883011:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55f95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.883032:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760762.883037:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760762.883046:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55f95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.883063:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55f95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.933752:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972898 is committed 00002000:00010000:3.0:1789760762.933933:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22847488 -> 22851584 00010000:00010000:3.0:1789760762.933956:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55f95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.933970:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55f95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.933980:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55f95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.933989:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760762.933995:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55f95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.934006:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55f95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.934038:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55f95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.934047:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe55f95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22847488->22851583] (req 22847488->22851583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.934196:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508397824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51261us (51558us total) trans 4294972898 rc 0/0 00000100:00100000:3.0:1789760762.934210:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5610 00000100:00100000:3.0:1789760762.934223:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741945297846 time 741987614641 state 0x2 ref 0 00000100:00100000:2.0:1789760762.938888:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760762.939166:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508397952 00000100:00100000:3.0:1789760762.939178:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508397952 00000100:00100000:3.0:1789760762.939267:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741945297846 time 741987614641 state 0x0 ref 1 00000100:00100000:3.0:1789760762.939324:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741987875057 time 741987614641 state 0x1 ref 1 00000100:00100000:3.0:1789760762.939353:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741987875057 time 742044063090 state 0x0 ref 1 00000100:00100000:3.0:1789760762.939367:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508397952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760762.939445:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22851584, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:3.0:1789760762.939459:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.939483:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760762.939490:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.939498:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.939517:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760762.992172:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972899 is committed 00002000:00010000:2.0:1789760762.992382:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22851584 -> 22855680 00010000:00010000:2.0:1789760762.992415:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.992431:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.992442:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.992450:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760762.992457:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.992467:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.992492:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760762.992501:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe55f9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22851584->22855679] (req 22851584->22855679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760762.992637:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508397952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53265us (53766us total) trans 4294972899 rc 0/0 00000100:00100000:2.0:1789760762.996346:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5611 00000100:00100000:2.0:1789760762.996362:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741987875057 time 742044063090 state 0x2 ref 0 00000100:00100000:3.0:1789760762.999438:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760762.999518:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508398080 00000100:00100000:1.0:1789760762.999532:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508398080 00000100:00100000:1.0:1789760762.999609:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 741987875057 time 742044063090 state 0x0 ref 1 00000100:00100000:1.0:1789760762.999629:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742044323506 time 742044063090 state 0x1 ref 1 00000100:00100000:1.0:1789760762.999802:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742044323506 time 742104510002 state 0x0 ref 1 00000100:00100000:1.0:1789760762.999827:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508398080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760762.999900:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22855680, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:1.0:1789760762.999921:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fa3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760762.999942:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760763.000071:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.000082:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fa3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.000102:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fa3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.050600:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22855680 -> 22859776 00010000:00010000:1.0:1789760763.050628:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fa3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.050644:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fa3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.050655:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fa3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.050664:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760763.050670:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fa3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.050680:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fa3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.050707:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fa3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.050715:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fa3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22855680->22859775] (req 22855680->22859775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.050999:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508398080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51101us (51505us total) trans 4294972900 rc 0/0 00000100:00100000:1.0:1789760763.051014:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5612 00000100:00100000:1.0:1789760763.051100:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742044323506 time 742104510002 state 0x2 ref 0 00000001:00080000:2.0:1789760763.051209:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972900 is committed 00000100:00100000:1.0:1789760763.058795:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760763.059110:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508398208 00000100:00100000:3.0:1789760763.059123:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508398208 00000100:00100000:3.0:1789760763.059346:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742044323506 time 742104510002 state 0x0 ref 1 00000100:00100000:3.0:1789760763.059369:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742104770418 time 742104510002 state 0x1 ref 1 00000100:00100000:3.0:1789760763.059405:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742104770418 time 742164115345 state 0x0 ref 1 00000100:00100000:3.0:1789760763.059422:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508398208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760763.059490:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22859776, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:3.0:1789760763.059502:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55faa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.059524:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760763.059529:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.059537:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55faa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.059553:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55faa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760763.061259:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760763.099849:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972901 is committed 00002000:00010000:3.0:1789760763.100195:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22859776 -> 22863872 00010000:00010000:3.0:1789760763.100217:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55faa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.100239:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55faa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.100249:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55faa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.100257:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760763.100274:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55faa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.100318:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55faa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.100348:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55faa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.100356:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55faa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22859776->22863871] (req 22859776->22863871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.100670:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508398208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41245us (41883us total) trans 4294972901 rc 0/0 00000100:00100000:3.0:1789760763.100682:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5613 00000100:00100000:3.0:1789760763.100695:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742104770418 time 742164115345 state 0x2 ref 0 00000100:00100000:3.0:1789760763.107258:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760763.107514:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508398336 00000100:00100000:1.0:1789760763.107528:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508398336 00000100:00100000:1.0:1789760763.107607:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742104770418 time 742164115345 state 0x0 ref 1 00000100:00100000:1.0:1789760763.107626:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742164375761 time 742164115345 state 0x1 ref 1 00000100:00100000:1.0:1789760763.107661:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742164375761 time 742212371609 state 0x0 ref 1 00000100:00100000:1.0:1789760763.107674:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d42850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508398336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760763.107742:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22863872, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:1.0:1789760763.107754:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fb1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.107774:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760763.107780:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.107787:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.107804:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.159239:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972902 is committed 00002000:00010000:3.0:1789760763.159516:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22863872 -> 22867968 00010000:00010000:3.0:1789760763.159536:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.159553:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fb1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.159564:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.159572:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760763.159578:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.159589:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fb1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.159616:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fb1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.159625:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe55fb1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22863872->22867967] (req 22863872->22867967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.159743:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d42850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508398336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52064us (52494us total) trans 4294972902 rc 0/0 00000100:00100000:3.0:1789760763.159756:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5614 00000100:00100000:3.0:1789760763.159768:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742164375761 time 742212371609 state 0x2 ref 0 00000100:00100000:3.0:1789760763.167420:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760763.167578:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508398464 00000100:00100000:1.0:1789760763.167591:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508398464 00000100:00100000:1.0:1789760763.167662:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742164375761 time 742212371609 state 0x0 ref 1 00000100:00100000:1.0:1789760763.167680:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742212632025 time 742212371609 state 0x1 ref 1 00000100:00100000:1.0:1789760763.167715:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742212632025 time 742272424764 state 0x0 ref 1 00000100:00100000:1.0:1789760763.167728:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508398464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760763.167791:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22867968, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:1.0:1789760763.167803:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55fb8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.167824:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760763.167831:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.167840:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55fb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.167858:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55fb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.221506:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972903 is committed 00002000:00010000:3.0:1789760763.221684:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22867968 -> 22872064 00010000:00010000:3.0:1789760763.221705:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55fb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.221720:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55fb8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.221729:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55fb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.221737:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760763.221742:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55fb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.221752:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55fb8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.221994:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55fb8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.222004:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe55fb8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22867968->22872063] (req 22867968->22872063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.222160:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508398464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54426us (54747us total) trans 4294972903 rc 0/0 00000100:00100000:3.0:1789760763.222175:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5615 00000100:00100000:3.0:1789760763.222188:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742212632025 time 742272424764 state 0x2 ref 0 00000100:00100000:1.0:1789760763.226679:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760763.226772:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508398592 00000100:00100000:3.0:1789760763.226785:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508398592 00000100:00100000:3.0:1789760763.226868:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742212632025 time 742272424764 state 0x0 ref 1 00000100:00100000:3.0:1789760763.226887:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742272685180 time 742272424764 state 0x1 ref 1 00000100:00100000:3.0:1789760763.226932:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742272685180 time 742331641939 state 0x0 ref 1 00000100:00100000:3.0:1789760763.226946:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508398592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760763.227007:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22872064, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:3.0:1789760763.227017:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fbf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.227036:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760763.227041:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.227048:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.227065:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.264145:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972904 is committed 00002000:00010000:3.0:1789760763.264725:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22872064 -> 22876160 00010000:00010000:3.0:1789760763.264746:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.264761:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fbf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.264772:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.264780:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760763.264785:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.264796:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fbf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.264821:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fbf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.264895:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fbf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22872064->22876159] (req 22872064->22876159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.265033:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508398592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38082us (38361us total) trans 4294972904 rc 0/0 00000100:00100000:3.0:1789760763.265046:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5616 00000100:00100000:3.0:1789760763.265059:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742272685180 time 742331641939 state 0x2 ref 0 00000100:00100000:2.0:1789760763.270385:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760763.271092:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508398720 00000100:00100000:3.0:1789760763.271108:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508398720 00000100:00100000:3.0:1789760763.271180:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742272685180 time 742331641939 state 0x0 ref 1 00000100:00100000:3.0:1789760763.271198:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742331902355 time 742331641939 state 0x1 ref 1 00000100:00100000:3.0:1789760763.271283:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742331902355 time 742375993027 state 0x0 ref 1 00000100:00100000:3.0:1789760763.271329:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508398720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760763.271396:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22876160, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:3.0:1789760763.271406:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55fc6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.271424:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760763.271429:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.271436:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55fc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.271454:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55fc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.309065:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972905 is committed 00002000:00010000:2.0:1789760763.309745:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22876160 -> 22880256 00010000:00010000:2.0:1789760763.309767:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55fc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.309783:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55fc6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.309794:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55fc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.309801:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760763.309806:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55fc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.309816:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55fc6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.309851:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55fc6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.309861:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe55fc6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22876160->22880255] (req 22876160->22880255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.310000:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508398720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38667us (39745us total) trans 4294972905 rc 0/0 00000100:00100000:2.0:1789760763.310012:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5617 00000100:00100000:2.0:1789760763.310025:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742331902355 time 742375993027 state 0x2 ref 0 00000100:00100000:2.0:1789760763.313755:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760763.313928:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508398848 00000100:00100000:3.0:1789760763.313941:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508398848 00000100:00100000:3.0:1789760763.314011:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742331902355 time 742375993027 state 0x0 ref 1 00000100:00100000:3.0:1789760763.314030:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742376253443 time 742375993027 state 0x1 ref 1 00000100:00100000:3.0:1789760763.314104:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742376253443 time 742418812791 state 0x0 ref 1 00000100:00100000:3.0:1789760763.314121:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508398848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760763.314176:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22880256, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:3.0:1789760763.314187:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fcd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.314206:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760763.314211:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.314218:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.314235:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.343875:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972906 is committed 00002000:00010000:3.0:1789760763.343932:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22880256 -> 22884352 00010000:00010000:3.0:1789760763.343954:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.343970:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fcd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.343981:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fcd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.343988:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760763.343994:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fcd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.344005:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fcd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.344030:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fcd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.344036:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fcd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22880256->22884351] (req 22880256->22884351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.344171:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508398848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30045us (30424us total) trans 4294972906 rc 0/0 00000100:00100000:3.0:1789760763.344183:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5618 00000100:00100000:3.0:1789760763.344196:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742376253443 time 742418812791 state 0x2 ref 0 00000100:00100000:1.0:1789760763.350666:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760763.350753:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508398976 00000100:00100000:3.0:1789760763.350766:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508398976 00000100:00100000:3.0:1789760763.350849:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742376253443 time 742418812791 state 0x0 ref 1 00000100:00100000:3.0:1789760763.350868:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742419073207 time 742418812791 state 0x1 ref 1 00000100:00100000:3.0:1789760763.350903:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742419073207 time 742455613338 state 0x0 ref 1 00000100:00100000:3.0:1789760763.350925:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508398976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760763.350987:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22884352, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:3.0:1789760763.350996:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fd4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.351014:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760763.351019:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.351026:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.351043:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.385150:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972907 is committed 00002000:00010000:1.0:1789760763.385443:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22884352 -> 22888448 00010000:00010000:1.0:1789760763.385465:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.385479:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fd4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.385489:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.385497:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760763.385503:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.385513:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fd4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.385546:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fd4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.385555:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe55fd4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22884352->22888447] (req 22884352->22888447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.385687:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508398976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34757us (35028us total) trans 4294972907 rc 0/0 00000100:00100000:1.0:1789760763.385699:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5619 00000100:00100000:1.0:1789760763.385711:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742419073207 time 742455613338 state 0x2 ref 0 00000100:00100000:1.0:1789760763.389726:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760763.390152:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508399104 00000100:00100000:2.0:1789760763.390166:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508399104 00000100:00100000:2.0:1789760763.390246:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742419073207 time 742455613338 state 0x0 ref 1 00000100:00100000:2.0:1789760763.390274:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742455873754 time 742455613338 state 0x1 ref 1 00000100:00100000:2.0:1789760763.390352:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742455873754 time 742495062012 state 0x0 ref 1 00000100:00100000:2.0:1789760763.390367:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508399104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760763.390432:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22888448, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:2.0:1789760763.390444:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55fdb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.390464:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760763.390469:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.390477:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55fdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.390494:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55fdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.426607:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972908 is committed 00002000:00010000:3.0:1789760763.426729:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22888448 -> 22892544 00010000:00010000:3.0:1789760763.426748:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55fdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.426764:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55fdb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.426773:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55fdb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.426781:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760763.426787:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55fdb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.426797:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55fdb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.426827:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55fdb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.426835:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe55fdb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22888448->22892543] (req 22888448->22892543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.426953:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508399104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36583us (37237us total) trans 4294972908 rc 0/0 00000100:00100000:3.0:1789760763.426963:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5620 00000100:00100000:3.0:1789760763.426975:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742455873754 time 742495062012 state 0x2 ref 0 00000100:00100000:3.0:1789760763.432162:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760763.432274:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508399232 00000100:00100000:1.0:1789760763.432327:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508399232 00000100:00100000:1.0:1789760763.432403:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742455873754 time 742495062012 state 0x0 ref 1 00000100:00100000:1.0:1789760763.432422:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742495322428 time 742495062012 state 0x1 ref 1 00000100:00100000:1.0:1789760763.432457:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742495322428 time 742537167840 state 0x0 ref 1 00000100:00100000:1.0:1789760763.432471:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508399232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760763.432547:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22892544, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:1.0:1789760763.432559:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55fe2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.432583:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760763.432589:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.432596:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55fe2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.432614:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55fe2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.459268:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972909 is committed 00002000:00010000:1.0:1789760763.459544:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22892544 -> 22896640 00010000:00010000:1.0:1789760763.459564:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55fe2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.459580:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55fe2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.459589:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55fe2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.459598:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760763.459604:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55fe2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.459615:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55fe2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.459639:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55fe2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.459649:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe55fe2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22892544->22896639] (req 22892544->22896639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.460533:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508399232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27283us (27604us total) trans 4294972909 rc 0/0 00000100:00100000:1.0:1789760763.460550:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5621 00000100:00100000:1.0:1789760763.460563:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742495322428 time 742537167840 state 0x2 ref 0 00000100:00100000:1.0:1789760763.463929:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760763.464099:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508399360 00000100:00100000:3.0:1789760763.464112:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508399360 00000100:00100000:3.0:1789760763.464186:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742495322428 time 742537167840 state 0x0 ref 1 00000100:00100000:3.0:1789760763.464205:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742537428256 time 742537167840 state 0x1 ref 1 00000100:00100000:3.0:1789760763.464237:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742537428256 time 742568947595 state 0x0 ref 1 00000100:00100000:3.0:1789760763.464251:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508399360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760763.464437:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22896640, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:3.0:1789760763.464447:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55fe9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.464473:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760763.464479:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.464495:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55fe9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.464512:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55fe9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760763.501838:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22896640 -> 22900736 00010000:00010000:1.0:1789760763.501866:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55fe9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.501883:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55fe9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.501893:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55fe9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.501899:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972910 is committed 00010000:00010000:1.0:1789760763.501900:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760763.501906:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55fe9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.501925:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55fe9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.501957:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55fe9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.501966:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe55fe9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22896640->22900735] (req 22896640->22900735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.502149:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508399360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37893us (38241us total) trans 4294972910 rc 0/0 00000100:00100000:1.0:1789760763.502160:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5622 00000100:00100000:1.0:1789760763.502172:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742537428256 time 742568947595 state 0x2 ref 0 00000100:00100000:0.0:1789760763.505656:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760763.505953:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508399488 00000100:00100000:3.0:1789760763.505967:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508399488 00000100:00100000:3.0:1789760763.506112:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742537428256 time 742568947595 state 0x0 ref 1 00000100:00100000:3.0:1789760763.506132:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742569208011 time 742568947595 state 0x1 ref 1 00000100:00100000:3.0:1789760763.506171:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742569208011 time 742610881543 state 0x0 ref 1 00000100:00100000:3.0:1789760763.506186:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508399488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760763.506255:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22900736, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0xb007 00010000:00010000:3.0:1789760763.506401:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55ff0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.506422:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760763.506428:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.506436:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55ff0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.506461:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55ff0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760763.553818:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972911 is committed 00002000:00010000:0.0:1789760763.553849:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22900736 -> 22904832 00002000:00010000:0.0:1789760763.553854:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760763 -> 1789760764 00002000:00010000:0.0:1789760763.553857:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760763 -> 1789760764 00010000:00010000:0.0:1789760763.553875:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55ff0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.553890:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55ff0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.553900:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55ff0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.553917:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760763.553923:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55ff0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.553934:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55ff0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.553959:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55ff0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.554059:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe55ff0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22900736->22904831] (req 22900736->22904831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.554180:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508399488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47991us (48533us total) trans 4294972911 rc 0/0 00000100:00100000:0.0:1789760763.554193:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5623 00000100:00100000:0.0:1789760763.554205:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742569208011 time 742610881543 state 0x2 ref 0 00000100:00100000:3.0:1789760763.557380:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760763.557487:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508399616 00000100:00100000:1.0:1789760763.557500:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508399616 00000100:00100000:1.0:1789760763.557565:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742569208011 time 742610881543 state 0x0 ref 1 00000100:00100000:1.0:1789760763.557583:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742611141959 time 742610881543 state 0x1 ref 1 00000100:00100000:1.0:1789760763.557650:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742611141959 time 742662360235 state 0x0 ref 1 00000100:00100000:1.0:1789760763.557664:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508399616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760763.557723:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22904832, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:1.0:1789760763.557731:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ff7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.557750:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760763.557755:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.557763:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ff7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.557779:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ff7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.611270:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972912 is committed 00002000:00010000:1.0:1789760763.611527:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22904832 -> 22908928 00010000:00010000:1.0:1789760763.611545:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ff7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.611560:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ff7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.611570:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ff7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.611577:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760763.611582:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ff7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.611593:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ff7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.611616:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ff7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.611629:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe55ff7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22904832->22908927] (req 22904832->22908927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.611845:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508399616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54176us (54472us total) trans 4294972912 rc 0/0 00000100:00100000:1.0:1789760763.611857:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5624 00000100:00100000:1.0:1789760763.611869:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742611141959 time 742662360235 state 0x2 ref 0 00000100:00100000:0.0:1789760763.615275:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760763.615351:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508399744 00000100:00100000:1.0:1789760763.615364:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508399744 00000100:00100000:1.0:1789760763.615478:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742611141959 time 742662360235 state 0x0 ref 1 00000100:00100000:1.0:1789760763.615510:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742662620651 time 742662360235 state 0x1 ref 1 00000100:00100000:1.0:1789760763.615543:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742662620651 time 742720252926 state 0x0 ref 1 00000100:00100000:1.0:1789760763.615557:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508399744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760763.615631:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22908928, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:1.0:1789760763.615642:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ffe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.615662:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760763.615667:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.615675:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ffe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.615702:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ffe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.647321:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760763.664803:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972913 is committed 00002000:00010000:2.0:1789760763.665116:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22908928 -> 22913024 00010000:00010000:2.0:1789760763.665136:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ffe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.665151:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ffe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.665162:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ffe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.665170:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760763.665176:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ffe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.665186:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ffe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.665212:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ffe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.665226:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe55ffe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22908928->22913023] (req 22908928->22913023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760763.665396:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508399744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49834us (50127us total) trans 4294972913 rc 0/0 00000100:00100000:2.0:1789760763.665413:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5625 00000100:00100000:2.0:1789760763.665426:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742662620651 time 742720252926 state 0x2 ref 0 00000100:00100000:0.0:1789760763.671528:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760763.671614:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508399872 00000100:00100000:3.0:1789760763.671627:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508399872 00000100:00100000:3.0:1789760763.671703:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742662620651 time 742720252926 state 0x0 ref 1 00000100:00100000:3.0:1789760763.671727:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742720513342 time 742720252926 state 0x1 ref 1 00000100:00100000:3.0:1789760763.671765:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742720513342 time 742776474814 state 0x0 ref 1 00000100:00100000:3.0:1789760763.671779:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508399872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760763.671846:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22913024, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:3.0:1789760763.671857:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56005 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.671876:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760763.671881:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.671889:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56005 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.672457:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56005 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760763.711108:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972914 is committed 00002000:00010000:0.0:1789760763.711239:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22913024 -> 22917120 00010000:00010000:0.0:1789760763.711361:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56005 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.711377:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56005 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.711387:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56005 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.711394:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760763.711400:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56005 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.711411:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56005 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.711437:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56005 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.711451:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56005 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22913024->22917119] (req 22913024->22917119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760763.711572:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508399872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39788us (40052us total) trans 4294972914 rc 0/0 00000100:00100000:0.0:1789760763.711584:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5626 00000100:00100000:0.0:1789760763.711597:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742720513342 time 742776474814 state 0x2 ref 0 00000100:00100000:1.0:1789760763.725802:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760763.725932:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508400000 00000100:00100000:2.0:1789760763.725945:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508400000 00000100:00100000:2.0:1789760763.726018:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742720513342 time 742776474814 state 0x0 ref 1 00000100:00100000:2.0:1789760763.726038:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742776735230 time 742776474814 state 0x1 ref 1 00000100:00100000:2.0:1789760763.726074:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742776735230 time 742830784054 state 0x0 ref 1 00000100:00100000:2.0:1789760763.726090:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508400000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760763.726156:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22917120, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:2.0:1789760763.726168:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5600c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.726188:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760763.726194:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.726202:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5600c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.726226:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5600c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.779747:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972915 is committed 00002000:00010000:1.0:1789760763.779916:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22917120 -> 22921216 00010000:00010000:1.0:1789760763.779937:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5600c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.779954:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5600c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.779964:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5600c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.779973:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760763.779979:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5600c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.779989:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5600c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.780015:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5600c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.780029:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5600c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22917120->22921215] (req 22917120->22921215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.780373:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508400000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54279us (54580us total) trans 4294972915 rc 0/0 00000100:00100000:1.0:1789760763.780385:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5627 00000100:00100000:1.0:1789760763.780398:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742776735230 time 742830784054 state 0x2 ref 0 00000100:00100000:1.0:1789760763.786416:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760763.786555:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508400128 00000100:00100000:2.0:1789760763.786567:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508400128 00000100:00100000:2.0:1789760763.786636:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742776735230 time 742830784054 state 0x0 ref 1 00000100:00100000:2.0:1789760763.786656:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742831044470 time 742830784054 state 0x1 ref 1 00000100:00100000:2.0:1789760763.786689:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742831044470 time 742891399449 state 0x0 ref 1 00000100:00100000:2.0:1789760763.786703:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508400128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760763.786769:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22921216, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:2.0:1789760763.786779:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56013 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.786797:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760763.786803:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760763.786810:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56013 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760763.786827:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56013 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.835364:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972916 is committed 00002000:00010000:1.0:1789760763.835564:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22921216 -> 22925312 00010000:00010000:1.0:1789760763.835584:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56013 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.835598:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56013 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.835607:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56013 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.835615:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760763.835621:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56013 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.835631:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56013 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.835662:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56013 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.835670:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56013 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22921216->22925311] (req 22921216->22925311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.835795:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508400128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49088us (49388us total) trans 4294972916 rc 0/0 00000100:00100000:1.0:1789760763.835807:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5628 00000100:00100000:1.0:1789760763.835819:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742831044470 time 742891399449 state 0x2 ref 0 00000100:00100000:1.0:1789760763.844976:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760763.845093:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508400256 00000100:00100000:0.0:1789760763.845109:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508400256 00000100:00100000:0.0:1789760763.845199:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742831044470 time 742891399449 state 0x0 ref 1 00000100:00100000:0.0:1789760763.845218:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742891659865 time 742891399449 state 0x1 ref 1 00000100:00100000:0.0:1789760763.845251:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742891659865 time 742949960464 state 0x0 ref 1 00000100:00100000:0.0:1789760763.845268:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508400256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760763.845375:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22925312, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:0.0:1789760763.845387:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5601a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.845407:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760763.845413:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760763.845420:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5601a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760763.845492:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5601a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.896978:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972917 is committed 00002000:00010000:1.0:1789760763.897152:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22925312 -> 22929408 00010000:00010000:1.0:1789760763.897173:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5601a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.897190:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5601a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.897200:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5601a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.897208:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760763.897215:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5601a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.897230:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5601a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.897270:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5601a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.897279:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5601a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22925312->22929407] (req 22925312->22929407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.897517:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508400256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52246us (52551us total) trans 4294972917 rc 0/0 00000100:00100000:1.0:1789760763.897530:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5629 00000100:00100000:1.0:1789760763.897543:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742891659865 time 742949960464 state 0x2 ref 0 00000100:00100000:3.0:1789760763.901170:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760763.902011:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508400384 00000100:00100000:1.0:1789760763.902027:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508400384 00000100:00100000:1.0:1789760763.902111:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742891659865 time 742949960464 state 0x0 ref 1 00000100:00100000:1.0:1789760763.902133:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742950220880 time 742949960464 state 0x1 ref 1 00000100:00100000:1.0:1789760763.902230:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742950220880 time 743006938513 state 0x0 ref 1 00000100:00100000:1.0:1789760763.902246:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508400384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760763.902354:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22929408, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:1.0:1789760763.902366:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56021 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.902400:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760763.902406:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.902567:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56021 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.902587:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56021 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760763.967034:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972918 is committed 00002000:00010000:1.0:1789760763.970596:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22929408 -> 22933504 00010000:00010000:1.0:1789760763.970623:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56021 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.970640:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56021 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.970649:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56021 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.970656:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760763.970662:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56021 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.970671:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56021 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.970698:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56021 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.970707:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56021 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22929408->22933503] (req 22929408->22933503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760763.970847:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508400384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 68596us (69686us total) trans 4294972918 rc 0/0 00000100:00100000:1.0:1789760763.970861:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5630 00000100:00100000:1.0:1789760763.970873:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742950220880 time 743006938513 state 0x2 ref 0 00000100:00100000:0.0:1789760763.974953:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760763.975072:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508400512 00000100:00100000:1.0:1789760763.975085:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508400512 00000100:00100000:1.0:1789760763.975167:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 742950220880 time 743006938513 state 0x0 ref 1 00000100:00100000:1.0:1789760763.975188:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743007198929 time 743006938513 state 0x1 ref 1 00000100:00100000:1.0:1789760763.975229:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743007198929 time 743079938661 state 0x0 ref 1 00000100:00100000:1.0:1789760763.975243:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508400512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760763.975360:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22933504, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:1.0:1789760763.975372:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56028 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.975392:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760763.975397:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760763.975405:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56028 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760763.975432:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56028 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.029185:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972919 is committed 00002000:00010000:1.0:1789760764.029474:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22933504 -> 22937600 00010000:00010000:1.0:1789760764.029494:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56028 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.029509:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56028 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.029519:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56028 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.029527:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760764.029533:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56028 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.029544:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56028 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.029570:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56028 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.029577:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56028 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22933504->22937599] (req 22933504->22937599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.029712:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508400512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54462us (54907us total) trans 4294972919 rc 0/0 00000100:00100000:1.0:1789760764.029726:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5631 00000100:00100000:1.0:1789760764.029738:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743007198929 time 743079938661 state 0x2 ref 0 00000100:00100000:0.0:1789760764.035723:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760764.035892:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508400640 00000100:00100000:1.0:1789760764.035915:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508400640 00000100:00100000:1.0:1789760764.035990:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743007198929 time 743079938661 state 0x0 ref 1 00000100:00100000:1.0:1789760764.036010:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743080199077 time 743079938661 state 0x1 ref 1 00000100:00100000:1.0:1789760764.036047:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743080199077 time 743140757171 state 0x0 ref 1 00000100:00100000:1.0:1789760764.036061:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508400640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760764.036239:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22937600, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:1.0:1789760764.036250:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5602f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.036365:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760764.036371:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.036379:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5602f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.036396:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5602f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.069900:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972920 is committed 00002000:00010000:1.0:1789760764.070176:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22937600 -> 22941696 00010000:00010000:1.0:1789760764.070196:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5602f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.070211:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5602f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.070221:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5602f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.070229:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760764.070235:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5602f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.070245:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5602f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.070270:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5602f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.070277:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5602f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22937600->22941695] (req 22937600->22941695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.070423:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508400640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34358us (34707us total) trans 4294972920 rc 0/0 00000100:00100000:1.0:1789760764.070439:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5632 00000100:00100000:1.0:1789760764.070450:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743080199077 time 743140757171 state 0x2 ref 0 00000100:00100000:0.0:1789760764.074128:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760764.074225:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508400768 00000100:00100000:1.0:1789760764.074239:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508400768 00000100:00100000:1.0:1789760764.074351:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743080199077 time 743140757171 state 0x0 ref 1 00000100:00100000:1.0:1789760764.074369:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743141017587 time 743140757171 state 0x1 ref 1 00000100:00100000:1.0:1789760764.074408:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743141017587 time 743179118545 state 0x0 ref 1 00000100:00100000:1.0:1789760764.074423:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508400768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760764.074487:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22941696, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:1.0:1789760764.074498:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56036 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.074518:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760764.074524:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.074531:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56036 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.074549:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56036 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.132813:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972921 is committed 00002000:00010000:3.0:1789760764.133084:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22941696 -> 22945792 00010000:00010000:3.0:1789760764.133105:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56036 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.133120:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56036 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.133130:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56036 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.133139:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760764.133144:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56036 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.133156:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56036 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.133182:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56036 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.133191:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56036 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22941696->22945791] (req 22941696->22945791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.133484:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508400768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59052us (59360us total) trans 4294972921 rc 0/0 00000100:00100000:3.0:1789760764.133497:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5633 00000100:00100000:3.0:1789760764.133510:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743141017587 time 743179118545 state 0x2 ref 0 00000100:00100000:2.0:1789760764.139882:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760764.140279:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508400896 00000100:00100000:1.0:1789760764.140463:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508400896 00000100:00100000:0.0:1789760764.140738:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743141017587 time 743179118545 state 0x0 ref 1 00000100:00100000:0.0:1789760764.140760:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743179378961 time 743179118545 state 0x1 ref 1 00000100:00100000:0.0:1789760764.140805:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743179378961 time 743245514552 state 0x0 ref 1 00000100:00100000:0.0:1789760764.140820:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508400896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760764.140883:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22945792, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:0.0:1789760764.140894:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5603d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.140925:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760764.140930:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.140938:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5603d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.140961:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5603d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.180528:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972922 is committed 00002000:00010000:3.0:1789760764.180663:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22945792 -> 22949888 00010000:00010000:3.0:1789760764.180683:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5603d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.180698:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5603d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.180709:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5603d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.180718:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760764.180723:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5603d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.180734:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5603d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.180759:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5603d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.180774:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5603d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22945792->22949887] (req 22945792->22949887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.180913:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508400896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40082us (41032us total) trans 4294972922 rc 0/0 00000100:00100000:3.0:1789760764.180925:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5634 00000100:00100000:3.0:1789760764.180938:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743179378961 time 743245514552 state 0x2 ref 0 00000100:00100000:0.0:1789760764.184594:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760764.184720:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508401024 00000100:00100000:2.0:1789760764.184734:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508401024 00000100:00100000:2.0:1789760764.184806:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743179378961 time 743245514552 state 0x0 ref 1 00000100:00100000:2.0:1789760764.184836:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743245774968 time 743245514552 state 0x1 ref 1 00000100:00100000:2.0:1789760764.184868:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743245774968 time 743289578123 state 0x0 ref 1 00000100:00100000:2.0:1789760764.184882:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508401024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760764.184967:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22949888, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:2.0:1789760764.184978:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56044 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.185000:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760764.185005:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.185014:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56044 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.185033:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56044 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.220540:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972923 is committed 00002000:00010000:2.0:1789760764.220651:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22949888 -> 22953984 00010000:00010000:2.0:1789760764.220671:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56044 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.220688:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56044 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.220699:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56044 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.220707:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760764.220713:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56044 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.220723:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56044 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.220753:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56044 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.220763:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56044 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22949888->22953983] (req 22949888->22953983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.221054:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508401024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36166us (36468us total) trans 4294972923 rc 0/0 00000100:00100000:2.0:1789760764.221067:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5635 00000100:00100000:2.0:1789760764.221078:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743245774968 time 743289578123 state 0x2 ref 0 00000100:00100000:2.0:1789760764.224963:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760764.225078:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508401152 00000100:00100000:3.0:1789760764.225091:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508401152 00000100:00100000:3.0:1789760764.225165:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743245774968 time 743289578123 state 0x0 ref 1 00000100:00100000:3.0:1789760764.225185:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743289838539 time 743289578123 state 0x1 ref 1 00000100:00100000:3.0:1789760764.225276:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743289838539 time 743329985457 state 0x0 ref 1 00000100:00100000:3.0:1789760764.225365:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508401152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760764.225431:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22953984, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:3.0:1789760764.225442:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5604b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.225464:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.225469:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.225477:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5604b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.225500:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5604b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.286648:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972924 is committed 00002000:00010000:2.0:1789760764.286783:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22953984 -> 22958080 00010000:00010000:2.0:1789760764.286807:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5604b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.286829:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5604b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.286840:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5604b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.286847:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760764.286853:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5604b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.286864:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5604b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.286894:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5604b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.286911:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5604b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22953984->22958079] (req 22953984->22958079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.287034:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508401152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61665us (62080us total) trans 4294972924 rc 0/0 00000100:00100000:2.0:1789760764.287046:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5636 00000100:00100000:2.0:1789760764.287060:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743289838539 time 743329985457 state 0x2 ref 0 00000100:00100000:0.0:1789760764.290537:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760764.290722:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508401280 00000100:00100000:3.0:1789760764.290735:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508401280 00000100:00100000:3.0:1789760764.290808:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743289838539 time 743329985457 state 0x0 ref 1 00000100:00100000:3.0:1789760764.290827:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743330245873 time 743329985457 state 0x1 ref 1 00000100:00100000:3.0:1789760764.290896:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743330245873 time 743395605644 state 0x0 ref 1 00000100:00100000:3.0:1789760764.290919:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508401280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760764.290990:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22958080, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:3.0:1789760764.291001:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56052 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.291021:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.291027:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.291035:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56052 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.291052:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56052 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.327531:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972925 is committed 00002000:00010000:3.0:1789760764.327935:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22958080 -> 22962176 00010000:00010000:3.0:1789760764.327955:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56052 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.327970:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56052 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.327981:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56052 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.327988:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760764.327994:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56052 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.328004:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56052 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.328031:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56052 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.328038:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56052 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22958080->22962175] (req 22958080->22962175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.328207:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508401280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37283us (37679us total) trans 4294972925 rc 0/0 00000100:00100000:3.0:1789760764.328225:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5637 00000100:00100000:3.0:1789760764.328238:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743330245873 time 743395605644 state 0x2 ref 0 00000100:00100000:0.0:1789760764.333311:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760764.333810:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508401408 00000100:00100000:2.0:1789760764.333824:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508401408 00000100:00100000:2.0:1789760764.333899:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743330245873 time 743395605644 state 0x0 ref 1 00000100:00100000:2.0:1789760764.333930:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743395866060 time 743395605644 state 0x1 ref 1 00000100:00100000:2.0:1789760764.333967:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743395866060 time 743438677598 state 0x0 ref 1 00000100:00100000:2.0:1789760764.333980:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508401408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760764.334050:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22962176, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:2.0:1789760764.334061:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56059 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.334081:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760764.334086:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.334093:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56059 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.334110:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56059 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.371407:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972926 is committed 00002000:00010000:1.0:1789760764.372404:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22962176 -> 22966272 00010000:00010000:1.0:1789760764.372429:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56059 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.372444:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56059 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.372454:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56059 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.372463:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760764.372468:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56059 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.372478:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56059 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.372502:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56059 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.372512:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56059 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22962176->22966271] (req 22962176->22966271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.372634:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508401408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38648us (39344us total) trans 4294972926 rc 0/0 00000100:00100000:1.0:1789760764.372646:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5638 00000100:00100000:1.0:1789760764.372659:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743395866060 time 743438677598 state 0x2 ref 0 00000100:00100000:3.0:1789760764.376245:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760764.376333:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508401536 00000100:00100000:2.0:1789760764.376346:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508401536 00000100:00100000:2.0:1789760764.376419:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743395866060 time 743438677598 state 0x0 ref 1 00000100:00100000:2.0:1789760764.376437:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743438938014 time 743438677598 state 0x1 ref 1 00000100:00100000:2.0:1789760764.376475:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743438938014 time 743481185970 state 0x0 ref 1 00000100:00100000:2.0:1789760764.376489:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508401536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760764.376553:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22966272, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:2.0:1789760764.376564:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56060 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.376583:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760764.376587:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.376595:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56060 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.376612:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56060 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.414462:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972927 is committed 00002000:00010000:2.0:1789760764.415104:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22966272 -> 22970368 00010000:00010000:2.0:1789760764.415124:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56060 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.415140:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56060 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.415151:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56060 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.415161:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760764.415168:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56060 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.415178:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56060 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.415203:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56060 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.415212:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56060 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22966272->22970367] (req 22966272->22970367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.415533:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508401536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39037us (39293us total) trans 4294972927 rc 0/0 00000100:00100000:2.0:1789760764.415555:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5639 00000100:00100000:2.0:1789760764.415569:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743438938014 time 743481185970 state 0x2 ref 0 00000100:00100000:2.0:1789760764.424761:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760764.424858:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508401664 00000100:00100000:3.0:1789760764.424871:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508401664 00000100:00100000:3.0:1789760764.424960:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743438938014 time 743481185970 state 0x0 ref 1 00000100:00100000:3.0:1789760764.424979:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743481446386 time 743481185970 state 0x1 ref 1 00000100:00100000:3.0:1789760764.425017:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743481446386 time 743529726815 state 0x0 ref 1 00000100:00100000:3.0:1789760764.425031:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508401664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760764.425096:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22970368, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:3.0:1789760764.425109:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56067 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.425129:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.425134:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.425142:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56067 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.425159:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56067 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.460762:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972928 is committed 00002000:00010000:2.0:1789760764.461026:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22970368 -> 22974464 00010000:00010000:2.0:1789760764.461046:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56067 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.461062:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56067 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.461073:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56067 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.461086:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760764.461091:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56067 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.461102:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56067 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.461127:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56067 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.461139:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56067 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22970368->22974463] (req 22970368->22974463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.461264:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508401664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36220us (36503us total) trans 4294972928 rc 0/0 00000100:00100000:2.0:1789760764.461277:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5640 00000100:00100000:2.0:1789760764.461339:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743481446386 time 743529726815 state 0x2 ref 0 00000100:00100000:1.0:1789760764.465576:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760764.465929:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508401792 00000100:00100000:2.0:1789760764.465943:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508401792 00000100:00100000:2.0:1789760764.466017:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743481446386 time 743529726815 state 0x0 ref 1 00000100:00100000:2.0:1789760764.466035:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743529987231 time 743529726815 state 0x1 ref 1 00000100:00100000:2.0:1789760764.466070:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743529987231 time 743570779717 state 0x0 ref 1 00000100:00100000:2.0:1789760764.466094:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508401792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760764.466159:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22974464, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0xb007 00010000:00010000:2.0:1789760764.466169:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5606e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.466189:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760764.466195:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.466203:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5606e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.466220:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5606e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760764.506130:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972929 is committed 00002000:00010000:3.0:1789760764.506981:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22974464 -> 22978560 00002000:00010000:3.0:1789760764.506988:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760764 -> 1789760765 00002000:00010000:3.0:1789760764.506991:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760764 -> 1789760765 00010000:00010000:3.0:1789760764.507008:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5606e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.507024:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5606e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.507033:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5606e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.507041:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760764.507048:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5606e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.507058:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5606e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.507084:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5606e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.507093:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5606e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22974464->22978559] (req 22974464->22978559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.507391:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508401792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41292us (41823us total) trans 4294972929 rc 0/0 00000100:00100000:3.0:1789760764.507409:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5641 00000100:00100000:3.0:1789760764.507421:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743529987231 time 743570779717 state 0x2 ref 0 00000100:00100000:3.0:1789760764.511955:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760764.512113:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508401920 00000100:00100000:2.0:1789760764.512126:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508401920 00000100:00100000:2.0:1789760764.512211:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743529987231 time 743570779717 state 0x0 ref 1 00000100:00100000:2.0:1789760764.512231:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743571040133 time 743570779717 state 0x1 ref 1 00000100:00100000:2.0:1789760764.512276:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743571040133 time 743616985786 state 0x0 ref 1 00000100:00100000:2.0:1789760764.512319:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508401920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760764.512388:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22978560, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:2.0:1789760764.512399:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56075 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.512424:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:2.0:1789760764.512429:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760764.512436:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56075 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.512452:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56075 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.542801:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972930 is committed 00002000:00010000:2.0:1789760764.542933:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22978560 -> 22982656 00010000:00010000:2.0:1789760764.542953:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56075 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.542968:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56075 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.542978:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56075 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.542986:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760764.542991:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56075 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.543002:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56075 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.543028:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56075 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.543037:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56075 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22978560->22982655] (req 22978560->22982655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.543158:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508401920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30864us (31212us total) trans 4294972930 rc 0/0 00000100:00100000:2.0:1789760764.543170:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5642 00000100:00100000:2.0:1789760764.543182:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743571040133 time 743616985786 state 0x2 ref 0 00000100:00100000:2.0:1789760764.549091:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760764.550466:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508402048 00000100:00100000:3.0:1789760764.550481:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508402048 00000100:00100000:3.0:1789760764.550564:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743571040133 time 743616985786 state 0x0 ref 1 00000100:00100000:3.0:1789760764.550584:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743617246202 time 743616985786 state 0x1 ref 1 00000100:00100000:3.0:1789760764.550671:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743617246202 time 743655381076 state 0x0 ref 1 00000100:00100000:3.0:1789760764.550687:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508402048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760764.550749:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22982656, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:3.0:1789760764.550759:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5607c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.550778:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.550783:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.550792:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5607c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.550809:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5607c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760764.586447:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22982656 -> 22986752 00010000:00010000:1.0:1789760764.586474:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5607c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.586498:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5607c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.586508:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5607c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.586515:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760764.586520:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5607c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.586532:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5607c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.586562:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5607c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.586570:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5607c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22982656->22986751] (req 22982656->22986751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.586619:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972931 is committed 00000100:00100000:1.0:1789760764.586742:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508402048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36051us (37659us total) trans 4294972931 rc 0/0 00000100:00100000:1.0:1789760764.586754:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5643 00000100:00100000:1.0:1789760764.586764:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743617246202 time 743655381076 state 0x2 ref 0 00000100:00100000:1.0:1789760764.593624:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760764.593723:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508402176 00000100:00100000:3.0:1789760764.593741:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508402176 00000100:00100000:3.0:1789760764.593824:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743617246202 time 743655381076 state 0x0 ref 1 00000100:00100000:3.0:1789760764.593843:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743655641492 time 743655381076 state 0x1 ref 1 00000100:00100000:3.0:1789760764.593876:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743655641492 time 743698585289 state 0x0 ref 1 00000100:00100000:3.0:1789760764.593889:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508402176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760764.593969:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22986752, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:3.0:1789760764.593981:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56083 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.594000:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.594005:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.594013:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56083 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.594030:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56083 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.637435:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22986752 -> 22990848 00010000:00010000:0.0:1789760764.637462:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56083 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.637478:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56083 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.637488:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56083 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.637497:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760764.637503:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56083 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.637513:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56083 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.637540:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56083 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.637550:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56083 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22986752->22990847] (req 22986752->22990847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.637672:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508402176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43777us (44054us total) trans 4294972932 rc 0/0 00000100:00100000:0.0:1789760764.637684:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5644 00000100:00100000:0.0:1789760764.637698:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743655641492 time 743698585289 state 0x2 ref 0 00000001:00080000:1.0:1789760764.638913:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972932 is committed 00000100:00100000:2.0:1789760764.644691:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760764.644806:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508402304 00000100:00100000:3.0:1789760764.644880:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508402304 00000100:00100000:3.0:1789760764.644964:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743655641492 time 743698585289 state 0x0 ref 1 00000100:00100000:3.0:1789760764.644987:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743698845705 time 743698585289 state 0x1 ref 1 00000100:00100000:3.0:1789760764.645062:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743698845705 time 743749771312 state 0x0 ref 1 00000100:00100000:3.0:1789760764.645077:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508402304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760764.645142:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22990848, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:3.0:1789760764.645153:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5608a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.645173:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.645179:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.645186:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5608a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.645203:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5608a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.670872:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760764.682641:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972933 is committed 00002000:00010000:2.0:1789760764.682795:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22990848 -> 22994944 00010000:00010000:2.0:1789760764.682880:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5608a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.682906:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5608a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.682918:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5608a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.682927:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760764.682933:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5608a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.682943:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5608a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.682969:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5608a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760764.682978:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5608a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22990848->22994943] (req 22990848->22994943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760764.684396:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508402304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39309us (39709us total) trans 4294972933 rc 0/0 00000100:00100000:2.0:1789760764.684480:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5645 00000100:00100000:2.0:1789760764.684493:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743698845705 time 743749771312 state 0x2 ref 0 00000100:00100000:2.0:1789760764.687644:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760764.687731:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508402432 00000100:00100000:1.0:1789760764.687745:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508402432 00000100:00100000:1.0:1789760764.687815:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743698845705 time 743749771312 state 0x0 ref 1 00000100:00100000:1.0:1789760764.687897:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743750031728 time 743749771312 state 0x1 ref 1 00000100:00100000:1.0:1789760764.687939:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743750031728 time 743792648460 state 0x0 ref 1 00000100:00100000:1.0:1789760764.687954:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508402432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760764.688018:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22994944, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:1.0:1789760764.688029:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56091 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.688048:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:1.0:1789760764.688053:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.688061:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56091 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.688077:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56091 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760764.726642:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972934 is committed 00002000:00010000:3.0:1789760764.726765:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22994944 -> 22999040 00010000:00010000:3.0:1789760764.726787:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56091 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.726802:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56091 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.726812:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56091 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.726832:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760764.726839:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56091 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.726850:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56091 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.726877:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56091 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.726886:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56091 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22994944->22999039] (req 22994944->22999039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.727086:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508402432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39127us (39449us total) trans 4294972934 rc 0/0 00000100:00100000:3.0:1789760764.727099:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5646 00000100:00100000:3.0:1789760764.727112:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743750031728 time 743792648460 state 0x2 ref 0 00000100:00100000:1.0:1789760764.730659:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760764.730974:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508402560 00000100:00100000:0.0:1789760764.730987:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508402560 00000100:00100000:0.0:1789760764.731058:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743750031728 time 743792648460 state 0x0 ref 1 00000100:00100000:0.0:1789760764.731078:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743792908876 time 743792648460 state 0x1 ref 1 00000100:00100000:0.0:1789760764.731114:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743792908876 time 743835824573 state 0x0 ref 1 00000100:00100000:0.0:1789760764.731128:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508402560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760764.731191:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 22999040, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:0.0:1789760764.731202:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56098 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.731222:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760764.731227:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.731237:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56098 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.731263:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56098 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.765771:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972935 is committed 00002000:00010000:1.0:1789760764.766219:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 22999040 -> 23003136 00010000:00010000:1.0:1789760764.766241:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56098 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.766265:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56098 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.766276:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56098 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.766301:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760764.766307:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56098 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.766319:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56098 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.766344:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56098 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.766351:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56098 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [22999040->23003135] (req 22999040->23003135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.766573:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508402560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35440us (35923us total) trans 4294972935 rc 0/0 00000100:00100000:1.0:1789760764.766587:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5647 00000100:00100000:1.0:1789760764.766600:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743792908876 time 743835824573 state 0x2 ref 0 00000100:00100000:1.0:1789760764.771004:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760764.771085:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508402688 00000100:00100000:3.0:1789760764.771099:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508402688 00000100:00100000:3.0:1789760764.771180:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743792908876 time 743835824573 state 0x0 ref 1 00000100:00100000:3.0:1789760764.771200:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743836084989 time 743835824573 state 0x1 ref 1 00000100:00100000:3.0:1789760764.771239:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743836084989 time 743875949312 state 0x0 ref 1 00000100:00100000:3.0:1789760764.771268:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508402688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760764.771355:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23003136, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:3.0:1789760764.771367:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5609f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.771387:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.771393:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.771524:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5609f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.771545:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5609f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.798607:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534079488:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760764.798718:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760764.799159:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534079488 00000100:00100000:2.0:1789760764.799180:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534079488 00000100:00100000:2.0:1789760764.799244:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:2.0:1789760764.799260:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534079488:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760764.799341:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789760764.799580:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534079488:12345-0@lo:41:osp-pre-1-0.0 Request processed in 321us (862us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760764.799589:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:0.0:1789760764.799783:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534079488:0@lo:41:osp-pre-1-0.0 00000001:00080000:2.0:1789760764.811517:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972936 is committed 00002000:00010000:3.0:1789760764.811582:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23003136 -> 23007232 00010000:00010000:3.0:1789760764.811603:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5609f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.811617:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5609f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.811626:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5609f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.811633:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760764.811638:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5609f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.811650:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5609f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.811676:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5609f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.811684:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5609f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23003136->23007231] (req 23003136->23007231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.811912:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508402688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40632us (40908us total) trans 4294972936 rc 0/0 00000100:00100000:3.0:1789760764.811925:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5648 00000100:00100000:3.0:1789760764.811937:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743836084989 time 743875949312 state 0x2 ref 0 00000100:00100000:0.0:1789760764.815019:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760764.815100:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508402816 00000100:00100000:3.0:1789760764.815113:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508402816 00000100:00100000:3.0:1789760764.815192:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743836084989 time 743875949312 state 0x0 ref 1 00000100:00100000:3.0:1789760764.815212:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743876209728 time 743875949312 state 0x1 ref 1 00000100:00100000:3.0:1789760764.815258:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743876209728 time 743919958588 state 0x0 ref 1 00000100:00100000:3.0:1789760764.815273:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508402816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760764.815362:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23007232, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:3.0:1789760764.815376:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe560a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.815397:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.815402:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.815410:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe560a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.815428:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe560a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760764.845153:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23007232 -> 23011328 00010000:00010000:0.0:1789760764.845181:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe560a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.845195:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe560a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.845204:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe560a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.845210:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760764.845216:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe560a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.845225:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe560a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.845267:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe560a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.845276:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe560a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23007232->23011327] (req 23007232->23011327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760764.845426:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508402816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30147us (30410us total) trans 4294972937 rc 0/0 00000100:00100000:0.0:1789760764.845438:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5649 00000100:00100000:0.0:1789760764.845449:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743876209728 time 743919958588 state 0x2 ref 0 00000100:00100000:1.0:1789760764.848014:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760764.848104:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508402944 00000100:00100000:0.0:1789760764.848119:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508402944 00000100:00100000:0.0:1789760764.848189:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743876209728 time 743919958588 state 0x0 ref 1 00000100:00100000:0.0:1789760764.848209:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743920219004 time 743919958588 state 0x1 ref 1 00000100:00100000:0.0:1789760764.848241:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743920219004 time 743952951302 state 0x0 ref 1 00000100:00100000:0.0:1789760764.848265:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508402944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760764.848356:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23011328, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:0.0:1789760764.848367:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe560ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.848387:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:0.0:1789760764.848393:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.848401:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe560ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.848418:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe560ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.849281:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972937 is committed 00000001:00080000:2.0:1789760764.878240:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972938 is committed 00002000:00010000:1.0:1789760764.878454:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23011328 -> 23015424 00010000:00010000:1.0:1789760764.878473:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe560ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.878488:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe560ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.878496:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe560ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.878504:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760764.878509:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe560ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.878519:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe560ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.878545:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe560ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.878554:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe560ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23011328->23015423] (req 23011328->23015423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.878680:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508402944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30411us (30671us total) trans 4294972938 rc 0/0 00000100:00100000:1.0:1789760764.878692:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5650 00000100:00100000:1.0:1789760764.878703:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743920219004 time 743952951302 state 0x2 ref 0 00000100:00100000:2.0:1789760764.881856:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760764.881982:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508403072 00000100:00100000:3.0:1789760764.881995:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508403072 00000100:00100000:3.0:1789760764.882058:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743920219004 time 743952951302 state 0x0 ref 1 00000100:00100000:3.0:1789760764.882076:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743953211718 time 743952951302 state 0x1 ref 1 00000100:00100000:3.0:1789760764.882105:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743953211718 time 743986816119 state 0x0 ref 1 00000100:00100000:3.0:1789760764.882118:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508403072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760764.882178:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23015424, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:3.0:1789760764.882189:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.882208:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=743s v=(0 0) 00010000:00010000:3.0:1789760764.882213:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.882221:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.882243:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.914735:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23015424 -> 23019520 00010000:00010000:3.0:1789760764.914762:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.914779:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.914790:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.914799:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760764.914806:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760764.914948:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972939 is committed 00010000:00010000:3.0:1789760764.915037:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.915067:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.915077:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23015424->23019519] (req 23015424->23019519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.915202:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508403072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33079us (33353us total) trans 4294972939 rc 0/0 00000100:00100000:3.0:1789760764.915215:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5651 00000100:00100000:3.0:1789760764.915229:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743953211718 time 743986816119 state 0x2 ref 0 00000100:00100000:2.0:1789760764.919134:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760764.919224:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508403200 00000100:00100000:0.0:1789760764.919238:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508403200 00000100:00100000:0.0:1789760764.919343:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743953211718 time 743986816119 state 0x0 ref 1 00000100:00100000:0.0:1789760764.919361:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743987076535 time 743986816119 state 0x1 ref 1 00000100:00100000:0.0:1789760764.919385:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743987076535 time 744024095372 state 0x0 ref 1 00000100:00100000:0.0:1789760764.919397:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508403200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760764.919461:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23019520, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:0.0:1789760764.919472:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe560bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.919492:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760764.919497:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760764.919505:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe560bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760764.919520:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe560bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.955142:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23019520 -> 23023616 00010000:00010000:3.0:1789760764.955169:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe560bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.955185:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe560bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.955195:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe560bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.955204:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760764.955209:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe560bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.955219:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe560bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.955244:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe560bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.955271:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe560bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23019520->23023615] (req 23019520->23023615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.955417:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508403200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36013us (36290us total) trans 4294972940 rc 0/0 00000100:00100000:3.0:1789760764.955429:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5652 00000100:00100000:3.0:1789760764.955442:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743987076535 time 744024095372 state 0x2 ref 0 00000001:00080000:1.0:1789760764.956734:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972940 is committed 00000100:00100000:0.0:1789760764.959021:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760764.959142:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508403328 00000100:00100000:1.0:1789760764.959157:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508403328 00000100:00100000:1.0:1789760764.959228:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 743987076535 time 744024095372 state 0x0 ref 1 00000100:00100000:1.0:1789760764.959482:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744024355788 time 744024095372 state 0x1 ref 1 00000100:00100000:1.0:1789760764.959546:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744024355788 time 744064254778 state 0x0 ref 1 00000100:00100000:1.0:1789760764.959561:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508403328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760764.959624:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23023616, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:1.0:1789760764.959635:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe560c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.959654:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760764.959659:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760764.959667:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe560c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.959683:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe560c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760764.992799:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23023616 -> 23027712 00010000:00010000:1.0:1789760764.992828:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe560c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.992843:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe560c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.992853:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe560c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.992862:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760764.992868:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe560c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.992878:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe560c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.992912:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe560c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760764.992921:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe560c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23023616->23027711] (req 23023616->23027711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760764.993045:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508403328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33478us (34030us total) trans 4294972941 rc 0/0 00000100:00100000:1.0:1789760764.993063:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5653 00000100:00100000:1.0:1789760764.993076:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744024355788 time 744064254778 state 0x2 ref 0 00000001:00080000:1.0:1789760764.993131:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972941 is committed 00000100:00100000:2.0:1789760764.996780:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760764.997147:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508403456 00000100:00100000:3.0:1789760764.997163:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508403456 00000100:00100000:3.0:1789760764.997243:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744024355788 time 744064254778 state 0x0 ref 1 00000100:00100000:3.0:1789760764.997271:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744064515194 time 744064254778 state 0x1 ref 1 00000100:00100000:3.0:1789760764.997514:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744064515194 time 744102221648 state 0x0 ref 1 00000100:00100000:3.0:1789760764.997531:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508403456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760764.997598:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23027712, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:3.0:1789760764.997609:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.997631:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760764.997635:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.997643:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.997665:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.027893:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972942 is committed 00002000:00010000:2.0:1789760765.027957:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23027712 -> 23031808 00010000:00010000:2.0:1789760765.028042:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.028060:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.028071:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.028079:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760765.028084:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.028095:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.028121:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.028130:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe560c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23027712->23031807] (req 23027712->23031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.028257:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508403456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30723us (31487us total) trans 4294972942 rc 0/0 00000100:00100000:2.0:1789760765.028272:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5654 00000100:00100000:2.0:1789760765.028364:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744064515194 time 744102221648 state 0x2 ref 0 00000100:00100000:2.0:1789760765.031011:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760765.031123:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508403584 00000100:00100000:3.0:1789760765.031137:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508403584 00000100:00100000:3.0:1789760765.031260:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744064515194 time 744102221648 state 0x0 ref 1 00000100:00100000:3.0:1789760765.031275:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744102482064 time 744102221648 state 0x1 ref 1 00000100:00100000:2.0:1789760765.031336:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744102482064 time 744136043138 state 0x0 ref 1 00000100:00100000:2.0:1789760765.031356:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508403584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760765.031419:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23031808, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:2.0:1789760765.031430:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe560d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760765.031445:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789760765.031450:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.031455:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.031462:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe560d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.031479:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe560d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.073653:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972943 is committed 00002000:00010000:2.0:1789760765.073925:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23031808 -> 23035904 00010000:00010000:2.0:1789760765.073941:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe560d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.073954:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe560d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.073969:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe560d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.073976:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760765.073980:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe560d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.073988:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe560d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.074007:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe560d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.074014:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe560d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23031808->23035903] (req 23031808->23035903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.074153:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508403584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42797us (43151us total) trans 4294972943 rc 0/0 00000100:00100000:2.0:1789760765.074163:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5655 00000100:00100000:2.0:1789760765.074173:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744102482064 time 744136043138 state 0x2 ref 0 00000100:00100000:0.0:1789760765.078170:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760765.078506:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508403712 00000100:00100000:1.0:1789760765.078517:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508403712 00000100:00100000:1.0:1789760765.078568:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744102482064 time 744136043138 state 0x0 ref 1 00000100:00100000:1.0:1789760765.078582:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744136303554 time 744136043138 state 0x1 ref 1 00000100:00100000:1.0:1789760765.078613:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744136303554 time 744183323627 state 0x0 ref 1 00000100:00100000:1.0:1789760765.078625:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508403712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760765.078672:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23035904, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:1.0:1789760765.078681:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe560d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.078697:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:1.0:1789760765.078701:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760765.078707:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe560d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.078722:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe560d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.133314:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972944 is committed 00002000:00010000:2.0:1789760765.133463:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23035904 -> 23040000 00010000:00010000:2.0:1789760765.133479:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe560d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.133498:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe560d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.133508:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe560d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.133515:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760765.133519:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe560d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.133528:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe560d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.133548:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe560d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.133555:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe560d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23035904->23039999] (req 23035904->23039999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.133657:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508403712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55029us (55491us total) trans 4294972944 rc 0/0 00000100:00100000:2.0:1789760765.133668:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5656 00000100:00100000:2.0:1789760765.133678:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744136303554 time 744183323627 state 0x2 ref 0 00000100:00100000:1.0:1789760765.138576:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760765.138705:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508403840 00000100:00100000:2.0:1789760765.138716:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508403840 00000100:00100000:2.0:1789760765.138911:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744136303554 time 744183323627 state 0x0 ref 1 00000100:00100000:2.0:1789760765.138928:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744183584043 time 744183323627 state 0x1 ref 1 00000100:00100000:2.0:1789760765.138958:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744183584043 time 744243668636 state 0x0 ref 1 00000100:00100000:2.0:1789760765.138972:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508403840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760765.139021:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23040000, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:2.0:1789760765.139030:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.139046:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.139051:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.139057:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.139071:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.182518:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972945 is committed 00002000:00010000:2.0:1789760765.182622:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23040000 -> 23044096 00010000:00010000:2.0:1789760765.182637:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.182650:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.182664:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.182672:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760765.182677:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.182686:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.182708:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.182716:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23040000->23044095] (req 23040000->23044095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.182929:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508403840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43955us (44357us total) trans 4294972945 rc 0/0 00000100:00100000:2.0:1789760765.182940:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5657 00000100:00100000:2.0:1789760765.182950:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744183584043 time 744243668636 state 0x2 ref 0 00000100:00100000:2.0:1789760765.187701:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760765.187939:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508403968 00000100:00100000:3.0:1789760765.187948:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508403968 00000100:00100000:3.0:1789760765.188002:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744183584043 time 744243668636 state 0x0 ref 1 00000100:00100000:3.0:1789760765.188018:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744243929052 time 744243668636 state 0x1 ref 1 00000100:00100000:3.0:1789760765.188047:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744243929052 time 744292757076 state 0x0 ref 1 00000100:00100000:3.0:1789760765.188059:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508403968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760765.188107:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23044096, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:3.0:1789760765.188116:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe560e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.188132:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760765.188137:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.188143:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe560e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.188159:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe560e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.227217:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972946 is committed 00002000:00010000:2.0:1789760765.227358:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23044096 -> 23048192 00010000:00010000:2.0:1789760765.227372:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe560e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.227393:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe560e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.227403:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe560e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.227411:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760765.227416:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe560e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.227426:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe560e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.227445:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe560e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.227453:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe560e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23044096->23048191] (req 23044096->23048191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.227536:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508403968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39474us (39840us total) trans 4294972946 rc 0/0 00000100:00100000:2.0:1789760765.227545:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5658 00000100:00100000:2.0:1789760765.227553:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744243929052 time 744292757076 state 0x2 ref 0 00000100:00100000:0.0:1789760765.232111:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760765.232202:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508404096 00000100:00100000:2.0:1789760765.232211:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508404096 00000100:00100000:2.0:1789760765.232691:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744243929052 time 744292757076 state 0x0 ref 1 00000100:00100000:2.0:1789760765.232707:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744293017492 time 744292757076 state 0x1 ref 1 00000100:00100000:2.0:1789760765.232885:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744293017492 time 744337594016 state 0x0 ref 1 00000100:00100000:2.0:1789760765.232909:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508404096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760765.233001:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23048192, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:2.0:1789760765.233010:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.233034:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.233038:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.233045:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.233059:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.281489:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972947 is committed 00002000:00010000:0.0:1789760765.281986:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23048192 -> 23052288 00010000:00010000:0.0:1789760765.282002:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.282015:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.282023:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.282030:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760765.282036:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.282044:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.282064:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.282072:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe560ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23048192->23052287] (req 23048192->23052287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.282208:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508404096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49307us (50102us total) trans 4294972947 rc 0/0 00000100:00100000:0.0:1789760765.282218:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5659 00000100:00100000:0.0:1789760765.282227:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744293017492 time 744337594016 state 0x2 ref 0 00000100:00100000:0.0:1789760765.288512:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760765.288598:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508404224 00000100:00100000:2.0:1789760765.288607:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508404224 00000100:00100000:2.0:1789760765.288911:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744293017492 time 744337594016 state 0x0 ref 1 00000100:00100000:2.0:1789760765.288929:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744337854432 time 744337594016 state 0x1 ref 1 00000100:00100000:2.0:1789760765.289004:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744337854432 time 744393714668 state 0x0 ref 1 00000100:00100000:2.0:1789760765.289021:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508404224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760765.289068:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23052288, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:2.0:1789760765.289077:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe560f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.289093:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.289097:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.289103:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe560f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.289117:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe560f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760765.336600:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23052288 -> 23056384 00010000:00010000:2.0:1789760765.336691:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe560f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.336705:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe560f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.336713:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe560f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.336720:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760765.336795:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe560f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.336881:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe560f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.336912:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe560f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.336920:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe560f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23052288->23056383] (req 23052288->23056383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.337072:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972948 is committed 00000100:00100000:2.0:1789760765.337127:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508404224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48104us (48619us total) trans 4294972948 rc 0/0 00000100:00100000:2.0:1789760765.337139:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5660 00000100:00100000:2.0:1789760765.337150:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744337854432 time 744393714668 state 0x2 ref 0 00000100:00100000:2.0:1789760765.342126:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760765.342218:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508404352 00000100:00100000:3.0:1789760765.342227:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508404352 00000100:00100000:3.0:1789760765.342493:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744337854432 time 744393714668 state 0x0 ref 1 00000100:00100000:3.0:1789760765.342509:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744393975084 time 744393714668 state 0x1 ref 1 00000100:00100000:3.0:1789760765.342572:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744393975084 time 744447281626 state 0x0 ref 1 00000100:00100000:3.0:1789760765.342586:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508404352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760765.342642:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23056384, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:3.0:1789760765.342652:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe560fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.342668:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760765.342673:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.342679:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe560fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.342694:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe560fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.380108:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972949 is committed 00002000:00010000:3.0:1789760765.380576:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23056384 -> 23060480 00010000:00010000:3.0:1789760765.380591:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe560fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.380603:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe560fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.380611:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe560fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.380618:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760765.380692:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe560fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.380703:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe560fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.383883:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe560fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.383919:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe560fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23056384->23060479] (req 23056384->23060479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.384081:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508404352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41492us (41960us total) trans 4294972949 rc 0/0 00000100:00100000:3.0:1789760765.384091:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5661 00000100:00100000:3.0:1789760765.384102:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744393975084 time 744447281626 state 0x2 ref 0 00000100:00100000:0.0:1789760765.392948:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760765.393066:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508404480 00000100:00100000:2.0:1789760765.393075:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508404480 00000100:00100000:2.0:1789760765.393130:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744393975084 time 744447281626 state 0x0 ref 1 00000100:00100000:2.0:1789760765.393145:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744447542042 time 744447281626 state 0x1 ref 1 00000100:00100000:2.0:1789760765.393213:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744447542042 time 744497922689 state 0x0 ref 1 00000100:00100000:2.0:1789760765.393227:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508404480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760765.393459:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23060480, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:2.0:1789760765.393469:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56101 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.393485:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.393489:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.393496:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56101 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.393510:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56101 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760765.436745:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972950 is committed 00002000:00010000:3.0:1789760765.437138:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23060480 -> 23064576 00010000:00010000:3.0:1789760765.437154:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56101 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.437177:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56101 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.437186:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56101 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.437193:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760765.437198:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56101 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.437207:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56101 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.437227:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56101 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.437234:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56101 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23060480->23064575] (req 23060480->23064575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.437441:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508404480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44211us (44497us total) trans 4294972950 rc 0/0 00000100:00100000:3.0:1789760765.437452:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5662 00000100:00100000:3.0:1789760765.437462:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744447542042 time 744497922689 state 0x2 ref 0 00000100:00100000:0.0:1789760765.442796:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760765.442999:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508404608 00000100:00100000:2.0:1789760765.443008:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508404608 00000100:00100000:2.0:1789760765.443064:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744447542042 time 744497922689 state 0x0 ref 1 00000100:00100000:2.0:1789760765.443079:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744498183105 time 744497922689 state 0x1 ref 1 00000100:00100000:2.0:1789760765.443110:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744498183105 time 744547820235 state 0x0 ref 1 00000100:00100000:2.0:1789760765.443122:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508404608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760765.443180:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23064576, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:2.0:1789760765.443190:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56108 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.443205:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.443210:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.443217:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56108 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.443231:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56108 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.476824:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972951 is committed 00002000:00010000:2.0:1789760765.476994:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23064576 -> 23068672 00010000:00010000:2.0:1789760765.477008:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56108 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.477021:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56108 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.477030:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56108 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.477038:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760765.477043:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56108 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.477052:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56108 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.477073:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56108 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.477082:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56108 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23064576->23068671] (req 23064576->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.478057:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508404608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34928us (35263us total) trans 4294972951 rc 0/0 00000100:00100000:2.0:1789760765.478069:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5663 00000100:00100000:2.0:1789760765.478079:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744498183105 time 744547820235 state 0x2 ref 0 00000100:00100000:0.0:1789760765.482940:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760765.483097:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508404736 00000100:00100000:3.0:1789760765.483106:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508404736 00000100:00100000:3.0:1789760765.483159:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744498183105 time 744547820235 state 0x0 ref 1 00000100:00100000:3.0:1789760765.483481:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744548080651 time 744547820235 state 0x1 ref 1 00000100:00100000:3.0:1789760765.483547:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744548080651 time 744588256888 state 0x0 ref 1 00000100:00100000:3.0:1789760765.483561:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508404736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760765.483611:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23068672, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0xb007 00010000:00010000:3.0:1789760765.483620:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5610f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.483635:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760765.483638:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.483644:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5610f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.483656:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5610f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.523613:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972952 is committed 00002000:00010000:2.0:1789760765.523721:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23068672 -> 23072768 00002000:00010000:2.0:1789760765.523726:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760765 -> 1789760766 00002000:00010000:2.0:1789760765.523728:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760765 -> 1789760766 00002000:00010000:2.0:1789760765.523730:0:8108:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:2.0:1789760765.523742:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5610f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.523754:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5610f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.523763:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5610f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.523770:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760765.523774:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5610f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.523782:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5610f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.523815:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5610f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.523824:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5610f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23068672->23072767] (req 23068672->23072767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.523923:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508404736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40360us (40987us total) trans 4294972952 rc 0/0 00000100:00100000:2.0:1789760765.523933:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5664 00000100:00100000:2.0:1789760765.523942:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744548080651 time 744588256888 state 0x2 ref 0 00000100:00100000:3.0:1789760765.528377:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760765.528532:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508404864 00000100:00100000:2.0:1789760765.528542:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508404864 00000100:00100000:2.0:1789760765.528593:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744548080651 time 744588256888 state 0x0 ref 1 00000100:00100000:2.0:1789760765.528607:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744588517304 time 744588256888 state 0x1 ref 1 00000100:00100000:2.0:1789760765.528628:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744588517304 time 744633339387 state 0x0 ref 1 00000100:00100000:2.0:1789760765.528641:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508404864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760765.528689:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23072768, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:2.0:1789760765.528698:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56116 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.528711:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.528716:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.528722:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56116 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.528735:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56116 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760765.560025:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972953 is committed 00002000:00010000:3.0:1789760765.560264:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23072768 -> 23076864 00010000:00010000:3.0:1789760765.560279:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56116 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.560307:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56116 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.560316:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56116 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.560323:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760765.560328:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56116 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.560396:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56116 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.560418:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56116 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.560425:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56116 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23072768->23076863] (req 23072768->23076863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.560516:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508404864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31871us (32142us total) trans 4294972953 rc 0/0 00000100:00100000:3.0:1789760765.560526:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5665 00000100:00100000:3.0:1789760765.560535:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744588517304 time 744633339387 state 0x2 ref 0 00000100:00100000:0.0:1789760765.565171:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760765.565281:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508404992 00000100:00100000:2.0:1789760765.565599:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508404992 00000100:00100000:2.0:1789760765.565652:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744588517304 time 744633339387 state 0x0 ref 1 00000100:00100000:2.0:1789760765.565668:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744633599803 time 744633339387 state 0x1 ref 1 00000100:00100000:2.0:1789760765.565699:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744633599803 time 744670408923 state 0x0 ref 1 00000100:00100000:2.0:1789760765.565710:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508404992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760765.565758:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23076864, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:2.0:1789760765.565768:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5611d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.565783:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.565787:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.565794:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5611d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.565908:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5611d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760765.596100:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23076864 -> 23080960 00010000:00010000:2.0:1789760765.596120:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5611d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.596133:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5611d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.596142:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5611d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.596149:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760765.596154:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5611d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.596163:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5611d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.596183:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5611d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.596192:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5611d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23076864->23080959] (req 23076864->23080959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.596390:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972954 is committed 00000100:00100000:2.0:1789760765.596604:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508404992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30890us (31438us total) trans 4294972954 rc 0/0 00000100:00100000:2.0:1789760765.596619:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5666 00000100:00100000:2.0:1789760765.596629:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744633599803 time 744670408923 state 0x2 ref 0 00000100:00100000:1.0:1789760765.599918:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760765.600061:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508405120 00000100:00100000:2.0:1789760765.600070:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508405120 00000100:00100000:2.0:1789760765.600122:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744633599803 time 744670408923 state 0x0 ref 1 00000100:00100000:2.0:1789760765.600137:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744670669339 time 744670408923 state 0x1 ref 1 00000100:00100000:2.0:1789760765.600171:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744670669339 time 744704881229 state 0x0 ref 1 00000100:00100000:2.0:1789760765.600184:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508405120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760765.600233:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23080960, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:2.0:1789760765.600477:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56124 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.600494:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.600499:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.600505:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56124 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.600519:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56124 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.629580:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23080960 -> 23085056 00010000:00010000:3.0:1789760765.629599:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56124 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.629611:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56124 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.629620:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56124 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.629627:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760765.629631:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56124 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.629640:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56124 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.629658:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56124 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.629665:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56124 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23080960->23085055] (req 23080960->23085055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.629759:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508405120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29572us (29844us total) trans 4294972955 rc 0/0 00000100:00100000:3.0:1789760765.629770:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5667 00000100:00100000:3.0:1789760765.629779:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744670669339 time 744704881229 state 0x2 ref 0 00000001:00080000:0.0:1789760765.629932:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972955 is committed 00000100:00100000:3.0:1789760765.633631:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760765.633765:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508405248 00000100:00100000:2.0:1789760765.633774:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508405248 00000100:00100000:2.0:1789760765.633885:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744670669339 time 744704881229 state 0x0 ref 1 00000100:00100000:2.0:1789760765.633910:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744705141645 time 744704881229 state 0x1 ref 1 00000100:00100000:2.0:1789760765.633939:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744705141645 time 744738649467 state 0x0 ref 1 00000100:00100000:2.0:1789760765.633952:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508405248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760765.633998:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23085056, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:2.0:1789760765.634007:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5612b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.634022:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:2.0:1789760765.634026:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760765.634033:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5612b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.634046:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5612b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.661308:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972956 is committed 00002000:00010000:2.0:1789760765.661515:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23085056 -> 23089152 00010000:00010000:2.0:1789760765.661529:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5612b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.661540:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5612b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.661549:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5612b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.661556:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760765.661561:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5612b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.661570:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5612b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.661590:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5612b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.661598:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5612b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23085056->23089151] (req 23085056->23089151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.661720:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508405248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27765us (28093us total) trans 4294972956 rc 0/0 00000100:00100000:2.0:1789760765.661729:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5668 00000100:00100000:2.0:1789760765.661739:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744705141645 time 744738649467 state 0x2 ref 0 00000100:00100000:2.0:1789760765.666139:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760765.666254:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508405376 00000100:00100000:3.0:1789760765.666264:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508405376 00000100:00100000:3.0:1789760765.666620:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744705141645 time 744738649467 state 0x0 ref 1 00000100:00100000:3.0:1789760765.666636:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744738909883 time 744738649467 state 0x1 ref 1 00000100:00100000:3.0:1789760765.666698:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744738909883 time 744771408460 state 0x0 ref 1 00000100:00100000:3.0:1789760765.666712:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508405376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760765.666760:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23089152, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:3.0:1789760765.666769:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56132 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.666784:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760765.666799:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.666812:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56132 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.666826:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56132 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.694461:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760765.696487:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972957 is committed 00002000:00010000:0.0:1789760765.696613:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23089152 -> 23093248 00010000:00010000:0.0:1789760765.696628:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56132 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.696640:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56132 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.696650:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56132 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.696663:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760765.696667:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56132 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.696676:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56132 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.696694:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56132 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.696703:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56132 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23089152->23093247] (req 23089152->23093247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.696793:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508405376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30079us (30659us total) trans 4294972957 rc 0/0 00000100:00100000:0.0:1789760765.696803:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5669 00000100:00100000:0.0:1789760765.696823:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744738909883 time 744771408460 state 0x2 ref 0 00000100:00100000:1.0:1789760765.701076:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760765.701211:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508405504 00000100:00100000:3.0:1789760765.701222:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508405504 00000100:00100000:3.0:1789760765.701774:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744738909883 time 744771408460 state 0x0 ref 1 00000100:00100000:3.0:1789760765.701792:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744771668876 time 744771408460 state 0x1 ref 1 00000100:00100000:3.0:1789760765.701875:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744771668876 time 744806584472 state 0x0 ref 1 00000100:00100000:3.0:1789760765.701901:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508405504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760765.701949:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23093248, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:3.0:1789760765.701958:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56139 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.701972:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760765.701976:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.701983:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56139 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.701997:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56139 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.736148:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972958 is committed 00002000:00010000:0.0:1789760765.736261:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23093248 -> 23097344 00010000:00010000:0.0:1789760765.736275:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56139 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.736307:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56139 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.736322:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56139 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.736330:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760765.736334:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56139 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.736344:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56139 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.736364:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56139 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.736373:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56139 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23093248->23097343] (req 23093248->23097343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.736500:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508405504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34609us (35430us total) trans 4294972958 rc 0/0 00000100:00100000:0.0:1789760765.736512:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5670 00000100:00100000:0.0:1789760765.736521:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744771668876 time 744806584472 state 0x2 ref 0 00000100:00100000:2.0:1789760765.739733:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760765.739826:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508405632 00000100:00100000:3.0:1789760765.739835:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508405632 00000100:00100000:3.0:1789760765.739890:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744771668876 time 744806584472 state 0x0 ref 1 00000100:00100000:3.0:1789760765.739922:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744806844888 time 744806584472 state 0x1 ref 1 00000100:00100000:3.0:1789760765.739953:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744806844888 time 744844663163 state 0x0 ref 1 00000100:00100000:3.0:1789760765.739966:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d41050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508405632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760765.740016:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23097344, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:3.0:1789760765.740024:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56140 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.740040:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760765.740045:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.740050:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56140 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.740064:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56140 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760765.771667:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23097344 -> 23101440 00010000:00010000:2.0:1789760765.771686:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56140 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.771703:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56140 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.771710:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56140 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.771717:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760765.771721:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56140 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.771731:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56140 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.771750:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56140 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760765.771758:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56140 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23097344->23101439] (req 23097344->23101439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760765.771855:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d41050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508405632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31887us (32128us total) trans 4294972959 rc 0/0 00000100:00100000:2.0:1789760765.771864:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5671 00000100:00100000:2.0:1789760765.771874:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744806844888 time 744844663163 state 0x2 ref 0 00000001:00080000:2.0:1789760765.772042:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972959 is committed 00000100:00100000:2.0:1789760765.775863:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760765.776065:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508405760 00000100:00100000:0.0:1789760765.776075:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508405760 00000100:00100000:0.0:1789760765.776129:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744806844888 time 744844663163 state 0x0 ref 1 00000100:00100000:0.0:1789760765.776143:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744844923579 time 744844663163 state 0x1 ref 1 00000100:00100000:0.0:1789760765.776174:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744844923579 time 744880884706 state 0x0 ref 1 00000100:00100000:0.0:1789760765.776185:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508405760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760765.776230:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23101440, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:0.0:1789760765.776239:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56147 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.776256:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760765.776260:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.776266:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56147 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.776280:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56147 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760765.811464:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23101440 -> 23105536 00010000:00010000:0.0:1789760765.811484:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56147 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.811490:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972960 is committed 00010000:00010000:0.0:1789760765.811600:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56147 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.811607:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56147 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.811612:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760765.811616:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56147 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.811623:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56147 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.811637:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56147 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.811642:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56147 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23101440->23105535] (req 23101440->23105535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760765.811715:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508405760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35527us (35857us total) trans 4294972960 rc 0/0 00000100:00100000:0.0:1789760765.811724:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5672 00000100:00100000:0.0:1789760765.811734:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744844923579 time 744880884706 state 0x2 ref 0 00000100:00100000:0.0:1789760765.814759:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760765.814922:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508405888 00000100:00100000:3.0:1789760765.814931:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508405888 00000100:00100000:3.0:1789760765.815047:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744844923579 time 744880884706 state 0x0 ref 1 00000100:00100000:3.0:1789760765.815062:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744881145122 time 744880884706 state 0x1 ref 1 00000100:00100000:3.0:1789760765.815118:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744881145122 time 744919827897 state 0x0 ref 1 00000100:00100000:3.0:1789760765.815131:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508405888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760765.815176:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23105536, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:3.0:1789760765.815185:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5614e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.815200:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760765.815205:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.815211:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5614e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.815226:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5614e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.836696:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23105536 -> 23109632 00010000:00010000:3.0:1789760765.836714:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5614e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.836728:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5614e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.836737:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5614e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.836745:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760765.836750:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5614e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.836759:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5614e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.836779:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5614e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.836787:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5614e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23105536->23109631] (req 23105536->23109631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.836903:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508405888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 21757us (22137us total) trans 4294972961 rc 0/0 00000100:00100000:3.0:1789760765.836914:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5673 00000100:00100000:3.0:1789760765.836924:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744881145122 time 744919827897 state 0x2 ref 0 00000001:00080000:2.0:1789760765.837083:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972961 is committed 00000100:00100000:2.0:1789760765.840484:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760765.840841:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508406016 00000100:00100000:3.0:1789760765.840851:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508406016 00000100:00100000:3.0:1789760765.840915:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744881145122 time 744919827897 state 0x0 ref 1 00000100:00100000:3.0:1789760765.840930:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744920088313 time 744919827897 state 0x1 ref 1 00000100:00100000:3.0:1789760765.840965:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744920088313 time 744945675600 state 0x0 ref 1 00000100:00100000:3.0:1789760765.840978:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508406016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760765.841025:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23109632, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:3.0:1789760765.841034:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56155 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.841051:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:3.0:1789760765.841055:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.841061:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56155 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.841075:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56155 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.888869:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23109632 -> 23113728 00010000:00010000:3.0:1789760765.888898:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56155 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.888911:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56155 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.888920:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56155 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.888927:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760765.888932:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56155 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.888941:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56155 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.888960:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56155 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.888968:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56155 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23109632->23113727] (req 23109632->23113727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.889061:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508406016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48081us (48581us total) trans 4294972962 rc 0/0 00000100:00100000:3.0:1789760765.889075:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5674 00000100:00100000:3.0:1789760765.889085:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744920088313 time 744945675600 state 0x2 ref 0 00000001:00080000:0.0:1789760765.890012:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972962 is committed 00000100:00100000:1.0:1789760765.892436:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760765.892556:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508406144 00000100:00100000:0.0:1789760765.892565:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508406144 00000100:00100000:0.0:1789760765.892620:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744920088313 time 744945675600 state 0x0 ref 1 00000100:00100000:0.0:1789760765.892636:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744945936016 time 744945675600 state 0x1 ref 1 00000100:00100000:0.0:1789760765.892666:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744945936016 time 744997376540 state 0x0 ref 1 00000100:00100000:0.0:1789760765.892678:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508406144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760765.892730:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23113728, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:0.0:1789760765.892740:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5615c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.892756:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=744s v=(0 0) 00010000:00010000:0.0:1789760765.892760:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760765.892766:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5615c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760765.892779:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5615c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760765.920567:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972963 is committed 00002000:00010000:1.0:1789760765.920725:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23113728 -> 23117824 00010000:00010000:1.0:1789760765.920739:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5615c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.920751:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5615c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.920759:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5615c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.920768:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760765.920772:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5615c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.920781:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5615c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.920810:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5615c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760765.920819:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5615c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23113728->23117823] (req 23113728->23117823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760765.920917:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508406144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28236us (28485us total) trans 4294972963 rc 0/0 00000100:00100000:1.0:1789760765.920927:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5675 00000100:00100000:1.0:1789760765.920937:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744945936016 time 744997376540 state 0x2 ref 0 00000100:00100000:1.0:1789760765.927523:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760765.927652:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508406272 00000100:00100000:3.0:1789760765.927663:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508406272 00000100:00100000:3.0:1789760765.927720:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744945936016 time 744997376540 state 0x0 ref 1 00000100:00100000:3.0:1789760765.927735:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744997636956 time 744997376540 state 0x1 ref 1 00000100:00100000:3.0:1789760765.927766:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744997636956 time 745032476881 state 0x0 ref 1 00000100:00100000:3.0:1789760765.927780:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508406272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760765.927834:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23117824, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:3.0:1789760765.927844:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56163 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.927858:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760765.927863:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.927868:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56163 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.927899:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56163 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.973363:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23117824 -> 23121920 00000001:00080000:2.0:1789760765.973431:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972964 is committed 00010000:00010000:3.0:1789760765.973491:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56163 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.973506:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56163 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.973516:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56163 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.973522:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760765.973527:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56163 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.973537:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56163 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.973558:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56163 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.973565:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56163 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23117824->23121919] (req 23117824->23121919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.973761:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508406272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45979us (46244us total) trans 4294972964 rc 0/0 00000100:00100000:3.0:1789760765.973772:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5676 00000100:00100000:3.0:1789760765.973789:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744997636956 time 745032476881 state 0x2 ref 0 00000100:00100000:2.0:1789760765.979607:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760765.979713:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508406400 00000100:00100000:3.0:1789760765.979726:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508406400 00000100:00100000:3.0:1789760765.979790:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 744997636956 time 745032476881 state 0x0 ref 1 00000100:00100000:3.0:1789760765.979817:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745032737297 time 745032476881 state 0x1 ref 1 00000100:00100000:3.0:1789760765.979855:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745032737297 time 745084564906 state 0x0 ref 1 00000100:00100000:3.0:1789760765.979868:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508406400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760765.979936:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23121920, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:3.0:1789760765.979947:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5616a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.979963:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760765.979968:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.979975:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5616a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.979992:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5616a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.019206:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972965 is committed 00002000:00010000:1.0:1789760766.019577:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23121920 -> 23126016 00010000:00010000:1.0:1789760766.019595:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5616a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.019610:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5616a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.019620:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5616a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.019628:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760766.019633:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5616a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.019642:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5616a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.019662:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5616a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.019682:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5616a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23121920->23126015] (req 23121920->23126015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.019791:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508406400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39921us (40189us total) trans 4294972965 rc 0/0 00000100:00100000:1.0:1789760766.019810:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5677 00000100:00100000:1.0:1789760766.019821:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745032737297 time 745084564906 state 0x2 ref 0 00000100:00100000:3.0:1789760766.022116:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760766.022275:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760766.022345:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760766.022485:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508406784 00000100:00100000:1.0:1789760766.022496:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508406784 00000100:00100000:1.0:1789760766.022567:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508407040 00000100:00100000:1.0:1789760766.022570:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508407040 00000100:00100000:1.0:1789760766.022602:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 939 00000100:00100000:1.0:1789760766.022612:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508406784:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760766.022687:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508406784:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 75us (412us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760766.022696:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 939 00000100:00100000:1.0:1789760766.022715:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 940 00000100:00100000:1.0:1789760766.022721:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508407040:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760766.022744:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508407040:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 24us (399us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760766.022751:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 940 00000100:00100000:3.0:1789760766.023015:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760766.023053:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760766.023074:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508406656 00000100:00100000:3.0:1789760766.023078:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508406656 00000100:00100000:3.0:1789760766.023101:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508406912 00000100:00100000:3.0:1789760766.023103:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508406912 00000100:00100000:3.0:1789760766.023123:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 941 00000100:00100000:3.0:1789760766.023129:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508406656:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:3.0:1789760766.023166:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508406656:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 37us (794us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.023173:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 941 00000100:00100000:3.0:1789760766.023185:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 942 00000100:00100000:3.0:1789760766.023191:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508406912:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:3.0:1789760766.023215:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508406912:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 24us (162us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.023221:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 942 00000100:00080000:0.0:1789760766.023322:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760766.023326:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789760766.024307:0:6459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508406528 00000100:00100000:2.0:1789760766.024315:0:6459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508406528 00000100:00100000:2.0:1789760766.024356:0:6459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 481 00000100:00100000:2.0:1789760766.024364:0:6459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988caf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2408:x1876699508406528:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760766.024470:0:6459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988caf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2408:x1876699508406528:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 106us (2363us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760766.024478:0:6459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 481 00000100:00100000:1.0:1789760766.026538:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760766.026683:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508407168 00000100:00100000:2.0:1789760766.026692:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508407168 00000100:00100000:2.0:1789760766.026740:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745032737297 time 745084564906 state 0x0 ref 1 00000100:00100000:2.0:1789760766.026754:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745084825322 time 745084564906 state 0x1 ref 1 00000100:00100000:2.0:1789760766.026780:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745084825322 time 745131490148 state 0x0 ref 1 00000100:00100000:2.0:1789760766.026792:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508407168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760766.026914:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23126016, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:2.0:1789760766.026923:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56171 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.026943:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760766.026947:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.026954:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56171 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.026967:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56171 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760766.079309:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23126016 -> 23130112 00010000:00010000:2.0:1789760766.079533:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56171 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.079547:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56171 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.079556:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56171 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.079563:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760766.079568:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56171 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.079577:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56171 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.079605:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56171 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.079614:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56171 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23126016->23130111] (req 23126016->23130111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.079872:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508407168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53078us (53336us total) trans 4294972966 rc 0/0 00000100:00100000:2.0:1789760766.079883:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5678 00000100:00100000:2.0:1789760766.079901:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745084825322 time 745131490148 state 0x2 ref 0 00000001:00080000:1.0:1789760766.079960:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972966 is committed 00000100:00100000:2.0:1789760766.083385:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760766.083497:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508407296 00000100:00100000:3.0:1789760766.083508:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508407296 00000100:00100000:3.0:1789760766.083564:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745084825322 time 745131490148 state 0x0 ref 1 00000100:00100000:3.0:1789760766.083579:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745131750564 time 745131490148 state 0x1 ref 1 00000100:00100000:3.0:1789760766.083604:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745131750564 time 745188315118 state 0x0 ref 1 00000100:00100000:3.0:1789760766.084181:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508407296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760766.084248:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23130112, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:3.0:1789760766.084258:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56178 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.084281:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760766.084317:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.084324:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56178 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.084348:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56178 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760766.144019:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23130112 -> 23134208 00010000:00010000:2.0:1789760766.144039:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56178 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.144051:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56178 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.144059:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56178 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.144066:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760766.144071:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56178 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.144081:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56178 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.144107:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56178 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.144114:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56178 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23130112->23134207] (req 23130112->23134207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.144217:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508407296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60597us (60834us total) trans 4294972967 rc 0/0 00000100:00100000:2.0:1789760766.144229:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5679 00000100:00100000:2.0:1789760766.144248:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745131750564 time 745188315118 state 0x2 ref 0 00000001:00080000:2.0:1789760766.148489:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972967 is committed 00000100:00100000:3.0:1789760766.153066:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760766.153230:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508407424 00000100:00100000:1.0:1789760766.153250:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508407424 00000100:00100000:1.0:1789760766.153344:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745131750564 time 745188315118 state 0x0 ref 1 00000100:00100000:1.0:1789760766.153360:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745188575534 time 745188315118 state 0x1 ref 1 00000100:00100000:1.0:1789760766.153389:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745188575534 time 745258100090 state 0x0 ref 1 00000100:00100000:1.0:1789760766.153401:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508407424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760766.153449:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23134208, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:1.0:1789760766.153458:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5617f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.153472:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.153476:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.153482:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5617f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.153496:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5617f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.187497:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972968 is committed 00002000:00010000:1.0:1789760766.187612:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23134208 -> 23138304 00010000:00010000:1.0:1789760766.187698:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5617f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.187712:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5617f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.187727:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5617f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.187734:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760766.187739:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5617f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.187748:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5617f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.187768:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5617f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.187776:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5617f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23134208->23138303] (req 23134208->23138303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.187880:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508407424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34475us (34819us total) trans 4294972968 rc 0/0 00000100:00100000:1.0:1789760766.187898:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5680 00000100:00100000:1.0:1789760766.187907:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745188575534 time 745258100090 state 0x2 ref 0 00000100:00100000:3.0:1789760766.191387:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760766.191471:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508407552 00000100:00100000:2.0:1789760766.191482:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508407552 00000100:00100000:2.0:1789760766.191535:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745188575534 time 745258100090 state 0x0 ref 1 00000100:00100000:2.0:1789760766.191549:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745258360506 time 745258100090 state 0x1 ref 1 00000100:00100000:2.0:1789760766.191581:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745258360506 time 745296291854 state 0x0 ref 1 00000100:00100000:2.0:1789760766.191593:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508407552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760766.191652:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23138304, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:2.0:1789760766.191661:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56186 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.191680:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760766.191684:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.191691:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56186 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.191706:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56186 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.233959:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972969 is committed 00002000:00010000:2.0:1789760766.234051:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23138304 -> 23142400 00010000:00010000:2.0:1789760766.234067:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56186 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.234081:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56186 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.234090:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56186 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.234098:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760766.234102:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56186 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.234112:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56186 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.234132:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56186 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.234141:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56186 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23138304->23142399] (req 23138304->23142399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.234374:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508407552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42777us (42991us total) trans 4294972969 rc 0/0 00000100:00100000:2.0:1789760766.234384:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5681 00000100:00100000:2.0:1789760766.234395:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745258360506 time 745296291854 state 0x2 ref 0 00000100:00100000:3.0:1789760766.238051:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760766.238131:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508407680 00000100:00100000:0.0:1789760766.238141:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508407680 00000100:00100000:0.0:1789760766.238193:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745258360506 time 745296291854 state 0x0 ref 1 00000100:00100000:0.0:1789760766.238208:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745296552270 time 745296291854 state 0x1 ref 1 00000100:00100000:0.0:1789760766.238249:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745296552270 time 745342958856 state 0x0 ref 1 00000100:00100000:0.0:1789760766.238262:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508407680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760766.238333:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23142400, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:0.0:1789760766.238343:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5618d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.238365:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760766.238369:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.238376:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5618d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.238390:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5618d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.270501:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972970 is committed 00002000:00010000:0.0:1789760766.270675:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23142400 -> 23146496 00010000:00010000:0.0:1789760766.270702:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5618d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.270714:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5618d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.270723:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5618d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.270730:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760766.270734:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5618d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.270742:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5618d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.270760:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5618d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.270768:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5618d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23142400->23146495] (req 23142400->23146495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.270868:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508407680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32603us (32822us total) trans 4294972970 rc 0/0 00000100:00100000:0.0:1789760766.270877:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5682 00000100:00100000:0.0:1789760766.270896:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745296552270 time 745342958856 state 0x2 ref 0 00000100:00100000:2.0:1789760766.274659:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760766.275175:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508407808 00000100:00100000:1.0:1789760766.275185:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508407808 00000100:00100000:1.0:1789760766.275247:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745296552270 time 745342958856 state 0x0 ref 1 00000100:00100000:1.0:1789760766.275262:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745343219272 time 745342958856 state 0x1 ref 1 00000100:00100000:1.0:1789760766.275316:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745343219272 time 745380025896 state 0x0 ref 1 00000100:00100000:1.0:1789760766.275337:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508407808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760766.275386:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23146496, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:1.0:1789760766.275395:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56194 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.275410:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.275413:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.275419:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56194 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.275432:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56194 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760766.305180:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23146496 -> 23150592 00010000:00010000:1.0:1789760766.305200:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56194 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.305213:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56194 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.305222:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56194 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.305230:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760766.305245:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56194 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.305256:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56194 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.305276:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56194 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.305302:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56194 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23146496->23150591] (req 23146496->23150591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.305394:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508407808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30062us (30739us total) trans 4294972971 rc 0/0 00000100:00100000:1.0:1789760766.305404:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5683 00000100:00100000:1.0:1789760766.305414:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745343219272 time 745380025896 state 0x2 ref 0 00000001:00080000:2.0:1789760766.306059:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972971 is committed 00000100:00100000:3.0:1789760766.314487:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760766.314805:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508407936 00000100:00100000:2.0:1789760766.314814:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508407936 00000100:00100000:2.0:1789760766.314867:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745343219272 time 745380025896 state 0x0 ref 1 00000100:00100000:2.0:1789760766.314882:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745380286312 time 745380025896 state 0x1 ref 1 00000100:00100000:2.0:1789760766.314929:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745380286312 time 745419639173 state 0x0 ref 1 00000100:00100000:2.0:1789760766.314941:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508407936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760766.314987:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23150592, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:2.0:1789760766.314996:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5619b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.315010:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760766.315014:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.315021:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5619b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.315046:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5619b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.369187:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972972 is committed 00002000:00010000:1.0:1789760766.369393:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23150592 -> 23154688 00010000:00010000:1.0:1789760766.369407:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5619b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.369419:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5619b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.369427:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5619b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.369438:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760766.369443:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5619b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.369451:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5619b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.369469:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5619b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.369476:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5619b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23150592->23154687] (req 23150592->23154687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.369567:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508407936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54624us (55086us total) trans 4294972972 rc 0/0 00000100:00100000:1.0:1789760766.369578:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5684 00000100:00100000:1.0:1789760766.369588:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745380286312 time 745419639173 state 0x2 ref 0 00000100:00100000:3.0:1789760766.373493:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760766.373664:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508408064 00000100:00100000:0.0:1789760766.373675:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508408064 00000100:00100000:0.0:1789760766.373728:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745380286312 time 745419639173 state 0x0 ref 1 00000100:00100000:0.0:1789760766.373744:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745419899589 time 745419639173 state 0x1 ref 1 00000100:00100000:0.0:1789760766.373772:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745419899589 time 745478483342 state 0x0 ref 1 00000100:00100000:0.0:1789760766.373783:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508408064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760766.373839:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23154688, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:0.0:1789760766.373848:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe561a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.373864:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:0.0:1789760766.373868:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760766.373874:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe561a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.373902:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe561a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760766.402748:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23154688 -> 23158784 00010000:00010000:1.0:1789760766.402768:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe561a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760766.402770:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972973 is committed 00010000:00010000:1.0:1789760766.402781:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe561a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.402797:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe561a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.402815:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760766.402819:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe561a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.402829:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe561a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.402849:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe561a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.402857:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe561a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23154688->23158783] (req 23154688->23158783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.402964:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508408064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29176us (29474us total) trans 4294972973 rc 0/0 00000100:00100000:1.0:1789760766.402974:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5685 00000100:00100000:1.0:1789760766.402984:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745419899589 time 745478483342 state 0x2 ref 0 00000100:00100000:2.0:1789760766.405790:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760766.406038:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508408192 00000100:00100000:1.0:1789760766.406047:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508408192 00000100:00100000:1.0:1789760766.406102:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745419899589 time 745478483342 state 0x0 ref 1 00000100:00100000:1.0:1789760766.406117:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745478743758 time 745478483342 state 0x1 ref 1 00000100:00100000:1.0:1789760766.406149:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745478743758 time 745510859525 state 0x0 ref 1 00000100:00100000:1.0:1789760766.406161:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508408192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760766.406209:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23158784, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:1.0:1789760766.406218:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe561a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.406232:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.406237:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.406243:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe561a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.406257:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe561a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.429232:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972974 is committed 00002000:00010000:2.0:1789760766.429453:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23158784 -> 23162880 00010000:00010000:2.0:1789760766.429468:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe561a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.429480:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe561a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.429489:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe561a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.429496:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760766.429501:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe561a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.429511:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe561a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.429531:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe561a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.429540:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe561a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23158784->23162879] (req 23158784->23162879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.429617:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508408192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23453us (23832us total) trans 4294972974 rc 0/0 00000100:00100000:2.0:1789760766.429626:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5686 00000100:00100000:2.0:1789760766.429635:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745478743758 time 745510859525 state 0x2 ref 0 00000100:00100000:3.0:1789760766.432122:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760766.432443:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508408320 00000100:00100000:1.0:1789760766.432452:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508408320 00000100:00100000:1.0:1789760766.432504:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745478743758 time 745510859525 state 0x0 ref 1 00000100:00100000:1.0:1789760766.432518:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745511119941 time 745510859525 state 0x1 ref 1 00000100:00100000:1.0:1789760766.432583:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745511119941 time 745537293862 state 0x0 ref 1 00000100:00100000:1.0:1789760766.432596:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508408320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760766.432645:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23162880, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:1.0:1789760766.432654:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe561b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.432668:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.432673:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.432679:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe561b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.432692:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe561b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.454089:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972975 is committed 00002000:00010000:2.0:1789760766.455567:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23162880 -> 23166976 00010000:00010000:2.0:1789760766.455586:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe561b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.455599:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe561b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.455607:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe561b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.455614:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760766.455618:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe561b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.455627:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe561b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.455653:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe561b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.455661:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe561b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23162880->23166975] (req 23162880->23166975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.455759:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508408320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23159us (23641us total) trans 4294972975 rc 0/0 00000100:00100000:2.0:1789760766.455769:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5687 00000100:00100000:2.0:1789760766.455779:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745511119941 time 745537293862 state 0x2 ref 0 00000100:00100000:3.0:1789760766.464663:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760766.464990:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508408448 00000100:00100000:1.0:1789760766.465000:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508408448 00000100:00100000:1.0:1789760766.465057:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745511119941 time 745537293862 state 0x0 ref 1 00000100:00100000:1.0:1789760766.465073:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745537554278 time 745537293862 state 0x1 ref 1 00000100:00100000:1.0:1789760766.465112:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745537554278 time 745569814191 state 0x0 ref 1 00000100:00100000:1.0:1789760766.465125:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508408448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760766.465173:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23166976, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:1.0:1789760766.465182:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe561b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.465198:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.465203:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.465209:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe561b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.465223:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe561b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.494659:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972976 is committed 00002000:00010000:2.0:1789760766.494760:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23166976 -> 23171072 00010000:00010000:2.0:1789760766.494775:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe561b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.494788:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe561b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.494796:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe561b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.494812:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760766.494817:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe561b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.494830:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe561b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.494849:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe561b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.494856:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe561b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23166976->23171071] (req 23166976->23171071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.494956:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508408448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29829us (30334us total) trans 4294972976 rc 0/0 00000100:00100000:2.0:1789760766.494966:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5688 00000100:00100000:2.0:1789760766.494976:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745537554278 time 745569814191 state 0x2 ref 0 00000100:00100000:0.0:1789760766.498614:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760766.498666:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508408576 00000100:00100000:2.0:1789760766.498675:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508408576 00000100:00100000:2.0:1789760766.498734:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745537554278 time 745569814191 state 0x0 ref 1 00000100:00100000:2.0:1789760766.498750:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745570074607 time 745569814191 state 0x1 ref 1 00000100:00100000:2.0:1789760766.498777:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745570074607 time 745603488174 state 0x0 ref 1 00000100:00100000:2.0:1789760766.498791:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508408576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760766.498843:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23171072, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0xb807 00010000:00010000:2.0:1789760766.498852:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe561be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.498867:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760766.498870:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.498874:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe561be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.498884:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe561be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.555211:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972977 is committed 00002000:00010000:1.0:1789760766.555433:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23171072 -> 23175168 00002000:00010000:1.0:1789760766.555439:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760766 -> 1789760767 00002000:00010000:1.0:1789760766.555441:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760766 -> 1789760767 00010000:00010000:1.0:1789760766.555455:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe561be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.555467:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe561be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.555477:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe561be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.555484:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760766.555488:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe561be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.555498:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe561be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.555519:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe561be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.555532:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe561be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23171072->23175167] (req 23171072->23175167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.555655:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508408576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56861us (57044us total) trans 4294972977 rc 0/0 00000100:00100000:1.0:1789760766.555665:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5689 00000100:00100000:1.0:1789760766.555674:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745570074607 time 745603488174 state 0x2 ref 0 00000100:00100000:3.0:1789760766.560875:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760766.561705:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508408704 00000100:00100000:2.0:1789760766.561790:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508408704 00000100:00100000:2.0:1789760766.561847:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745570074607 time 745603488174 state 0x0 ref 1 00000100:00100000:2.0:1789760766.561862:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745603748590 time 745603488174 state 0x1 ref 1 00000100:00100000:2.0:1789760766.561902:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745603748590 time 745666612715 state 0x0 ref 1 00000100:00100000:2.0:1789760766.561915:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508408704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760766.561965:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23175168, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:2.0:1789760766.561975:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe561c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.561992:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760766.561997:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.562004:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe561c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.562024:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe561c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.607438:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972978 is committed 00002000:00010000:1.0:1789760766.608969:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23175168 -> 23179264 00010000:00010000:1.0:1789760766.608987:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe561c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.609000:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe561c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.609008:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe561c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.609015:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760766.609020:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe561c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.609028:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe561c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.609053:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe561c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.609062:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe561c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23175168->23179263] (req 23175168->23179263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.609157:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508408704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47240us (48286us total) trans 4294972978 rc 0/0 00000100:00100000:1.0:1789760766.609166:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5690 00000100:00100000:1.0:1789760766.609175:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745603748590 time 745666612715 state 0x2 ref 0 00000100:00100000:3.0:1789760766.614595:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760766.614963:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508408832 00000100:00100000:2.0:1789760766.614972:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508408832 00000100:00100000:2.0:1789760766.615026:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745603748590 time 745666612715 state 0x0 ref 1 00000100:00100000:2.0:1789760766.615042:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745666873131 time 745666612715 state 0x1 ref 1 00000100:00100000:2.0:1789760766.615073:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745666873131 time 745719783890 state 0x0 ref 1 00000100:00100000:2.0:1789760766.615085:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508408832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760766.615135:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23179264, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:2.0:1789760766.615144:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe561cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.615158:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760766.615163:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.615169:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe561cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.615183:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe561cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760766.648124:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23179264 -> 23183360 00010000:00010000:2.0:1789760766.648146:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe561cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.648157:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe561cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.648165:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe561cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.648172:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760766.648177:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe561cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.648186:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe561cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.648206:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe561cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.648213:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe561cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23179264->23183359] (req 23179264->23183359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760766.648456:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508408832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33367us (33865us total) trans 4294972979 rc 0/0 00000100:00100000:2.0:1789760766.648467:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5691 00000100:00100000:2.0:1789760766.648478:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745666873131 time 745719783890 state 0x2 ref 0 00000001:00080000:0.0:1789760766.650156:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972979 is committed 00000100:00100000:0.0:1789760766.656608:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760766.656904:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508408960 00000100:00100000:1.0:1789760766.656913:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508408960 00000100:00100000:1.0:1789760766.656967:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745666873131 time 745719783890 state 0x0 ref 1 00000100:00100000:1.0:1789760766.656983:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745720044306 time 745719783890 state 0x1 ref 1 00000100:00100000:1.0:1789760766.657014:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745720044306 time 745761724617 state 0x0 ref 1 00000100:00100000:1.0:1789760766.657027:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508408960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760766.657074:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23183360, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:1.0:1789760766.657083:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe561d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.657099:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.657104:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.657110:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe561d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.657123:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe561d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.691901:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23183360 -> 23187456 00010000:00010000:3.0:1789760766.691921:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe561d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.691933:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe561d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.691941:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe561d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.691947:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760766.691952:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe561d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.691960:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe561d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.691978:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe561d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.691985:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe561d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23183360->23187455] (req 23183360->23187455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.692082:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508408960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35052us (35478us total) trans 4294972980 rc 0/0 00000100:00100000:3.0:1789760766.692094:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5692 00000100:00100000:3.0:1789760766.692104:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745720044306 time 745761724617 state 0x2 ref 0 00000001:00080000:0.0:1789760766.692310:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972980 is committed 00000100:00100000:1.0:1789760766.700204:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760766.700264:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508409088 00000100:00100000:2.0:1789760766.700274:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508409088 00000100:00100000:2.0:1789760766.700364:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745720044306 time 745761724617 state 0x0 ref 1 00000100:00100000:2.0:1789760766.700381:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745761985033 time 745761724617 state 0x1 ref 1 00000100:00100000:2.0:1789760766.700441:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745761985033 time 745805150814 state 0x0 ref 1 00000100:00100000:2.0:1789760766.700455:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508409088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760766.700563:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23187456, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:2.0:1789760766.700573:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe561da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.700588:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:2.0:1789760766.700593:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760766.700599:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe561da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760766.700613:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe561da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760766.718520:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760766.735162:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972981 is committed 00002000:00010000:3.0:1789760766.735342:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23187456 -> 23191552 00010000:00010000:3.0:1789760766.735357:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe561da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.735371:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe561da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.735380:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe561da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.735389:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760766.735394:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe561da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.735403:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe561da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.735425:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe561da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.735434:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe561da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23187456->23191551] (req 23187456->23191551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.735566:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508409088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35107us (35366us total) trans 4294972981 rc 0/0 00000100:00100000:3.0:1789760766.735577:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5693 00000100:00100000:3.0:1789760766.735586:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745761985033 time 745805150814 state 0x2 ref 0 00000100:00100000:2.0:1789760766.744446:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760766.744548:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508409216 00000100:00100000:1.0:1789760766.744558:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508409216 00000100:00100000:1.0:1789760766.744612:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745761985033 time 745805150814 state 0x0 ref 1 00000100:00100000:1.0:1789760766.744628:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745805411230 time 745805150814 state 0x1 ref 1 00000100:00100000:1.0:1789760766.744657:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745805411230 time 745849367544 state 0x0 ref 1 00000100:00100000:1.0:1789760766.744669:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508409216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760766.744717:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23191552, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:1.0:1789760766.744726:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe561e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.744741:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.744746:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.744753:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe561e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.744767:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe561e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760766.771505:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972982 is committed 00002000:00010000:3.0:1789760766.771871:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23191552 -> 23195648 00010000:00010000:3.0:1789760766.771895:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe561e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.771908:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe561e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.771918:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe561e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.771925:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760766.771930:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe561e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.771939:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe561e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.771961:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe561e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.771969:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe561e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23191552->23195647] (req 23191552->23195647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.772106:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508409216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27433us (27664us total) trans 4294972982 rc 0/0 00000100:00100000:3.0:1789760766.772115:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5694 00000100:00100000:3.0:1789760766.772125:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745805411230 time 745849367544 state 0x2 ref 0 00000100:00100000:3.0:1789760766.780880:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760766.781279:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508409344 00000100:00100000:1.0:1789760766.781326:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508409344 00000100:00100000:1.0:1789760766.781384:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745805411230 time 745849367544 state 0x0 ref 1 00000100:00100000:1.0:1789760766.781400:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745849627960 time 745849367544 state 0x1 ref 1 00000100:00100000:1.0:1789760766.781430:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745849627960 time 745886141196 state 0x0 ref 1 00000100:00100000:1.0:1789760766.781443:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508409344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760766.781492:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23195648, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:1.0:1789760766.781500:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.781516:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.781520:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.781526:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.781539:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.826595:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972983 is committed 00002000:00010000:1.0:1789760766.826690:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23195648 -> 23199744 00010000:00010000:1.0:1789760766.826706:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.826727:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.826737:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.826744:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760766.826749:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.826757:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.826780:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.826786:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23195648->23199743] (req 23195648->23199743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.826927:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508409344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45482us (46053us total) trans 4294972983 rc 0/0 00000100:00100000:1.0:1789760766.826938:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5695 00000100:00100000:1.0:1789760766.826948:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745849627960 time 745886141196 state 0x2 ref 0 00000100:00100000:1.0:1789760766.838770:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760766.838861:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508409472 00000100:00100000:3.0:1789760766.838873:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508409472 00000100:00100000:3.0:1789760766.838943:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745849627960 time 745886141196 state 0x0 ref 1 00000100:00100000:3.0:1789760766.838960:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745886401612 time 745886141196 state 0x1 ref 1 00000100:00100000:3.0:1789760766.838993:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745886401612 time 745943703073 state 0x0 ref 1 00000100:00100000:3.0:1789760766.839006:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508409472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760766.839057:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23199744, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:3.0:1789760766.839067:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe561ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.839085:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:3.0:1789760766.839090:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.839096:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe561ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.839110:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe561ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760766.847124:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888046d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534080000:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.847142:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888045c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3473:1876699534079616:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.847222:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888045880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534079744:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.854382:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760766.854449:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888044380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534080128:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.854496:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760766.854520:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888045500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534080640:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.854545:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760766.854566:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888047800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534081152:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.854588:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760766.854616:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534081664:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760766.854639:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760766.854659:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789760766.854667:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534081536:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760766.854756:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888044000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534079872:0@lo:400:kworker.0 00000100:00100000:1.0:1789760766.854759:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534079616 00000100:00100000:1.0:1789760766.854767:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534079616 00000100:00100000:3.0:1789760766.854796:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760766.854812:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 482 00000100:00100000:3.0:1789760766.854817:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534080256:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760766.854821:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cae450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3473:x1876699534079616:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.854843:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534080128 00000100:00100000:3.0:1789760766.854847:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534080128 00000100:00100000:1.0:1789760766.854871:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cae450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3473:x1876699534079616:12345-0@lo:400:kworker.0 Request processed in 51us (496us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.854871:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534079872 00000100:00100000:3.0:1789760766.854873:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534079872 00000100:00100000:1.0:1789760766.854878:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 482 00000100:00100000:3.0:1789760766.854897:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:3.0:1789760766.854903:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534080128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.854938:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534080128:12345-0@lo:400:kworker.0 Request processed in 35us (442us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.854944:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00100000:3.0:1789760766.854956:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 451 00000100:00100000:3.0:1789760766.854962:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534079872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.854967:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760766.855018:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534081664 00000100:00100000:1.0:1789760766.855022:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534081664 00000100:00100000:1.0:1789760766.855053:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534081536 00000100:00100000:1.0:1789760766.855056:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534081536 00000100:00100000:0.0:1789760766.855086:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760766.855093:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 388 00000100:00100000:1.0:1789760766.855099:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534081664:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760766.855115:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3835691008 avail 3787456512 00000100:00100000:1.0:1789760766.855147:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534081664:12345-0@lo:13:osp-pre-0-1.0 Request processed in 48us (507us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760766.855153:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 388 00000100:00080000:0.0:1789760766.855160:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760766.855163:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00080000:0.0:1789760766.855164:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760766.855169:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534081536:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760766.855177:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534080000 00000100:00100000:0.0:1789760766.855180:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534080000 00002000:00000024:1.0:1789760766.855190:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760766.855203:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 943 00000100:00100000:0.0:1789760766.855209:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534080000:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760766.855214:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534081536:12345-0@lo:13:osp-pre-1-0.0 Request processed in 46us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760766.855219:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00000100:00100000:0.0:1789760766.855339:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534080000:12345-0@lo:400:kworker.0 Request processed in 129us (253us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760766.855346:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 943 00000100:00100000:0.0:1789760766.855381:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888046d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534080000:0@lo:400:kworker.0 00000100:00100000:0.0:1789760766.855389:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888046a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3472:1876699534080512:0@lo:400:kworker.0 00000100:00100000:0.0:1789760766.855411:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760766.855431:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888045180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3472:1876699534081024:0@lo:400:kworker.0 00000100:00100000:0.0:1789760766.855452:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760766.855491:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534080512 00000100:00100000:0.0:1789760766.855494:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534080512 00000100:00100000:0.0:1789760766.855519:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534081024 00000100:00100000:0.0:1789760766.855522:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534081024 00000100:00100000:0.0:1789760766.855542:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 944 00000100:00100000:0.0:1789760766.855548:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3472:x1876699534080512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.855666:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534079872:12345-0@lo:400:kworker.0 Request processed in 703us (868us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.855673:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 451 00000100:00100000:3.0:1789760766.855696:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 945 00000100:00100000:3.0:1789760766.855704:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3472:x1876699534081024:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.855740:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3472:x1876699534081024:12345-0@lo:400:kworker.0 Request processed in 37us (286us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.855746:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 945 00000100:00100000:0.0:1789760766.855840:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3472:x1876699534080512:12345-0@lo:400:kworker.0 Request processed in 291us (427us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760766.855845:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 944 00000100:00080000:0.0:1789760766.855862:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760766.859341:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.859398:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888044700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3475:1876699534080896:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.859453:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.859474:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534081408:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760766.859503:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.859555:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888044000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534079872:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.859855:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534080256 00000100:00100000:3.0:1789760766.859862:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534080256 00000100:00100000:3.0:1789760766.859920:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534081408 00000100:00100000:3.0:1789760766.859924:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534081408 00000100:00100000:3.0:1789760766.859942:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:3.0:1789760766.859951:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534080256:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760766.859974:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760766.860018:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534080256:12345-0@lo:13:osp-pre-1-1.0 Request processed in 68us (683us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.860025:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00100000:3.0:1789760766.860039:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 391 00000100:00100000:3.0:1789760766.860044:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534081408:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760766.860054:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3835691008 avail 3787456512 00000100:00100000:3.0:1789760766.860080:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534081408:12345-0@lo:13:osp-pre-0-0.0 Request processed in 36us (576us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.860085:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 391 00000100:00100000:3.0:1789760766.860102:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534080896 00000100:00100000:3.0:1789760766.860105:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534080896 00000100:00100000:3.0:1789760766.860134:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 946 00000100:00100000:3.0:1789760766.860140:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3475:x1876699534080896:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.860171:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3475:x1876699534080896:12345-0@lo:400:kworker.0 Request processed in 32us (717us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.860177:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 946 00000100:00080000:3.0:1789760766.860191:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760766.860196:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760766.860198:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760766.860229:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534080256:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760766.860254:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888044700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3475:1876699534080896:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.860268:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fc9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534081408:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760766.860334:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.860357:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888045f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534080384:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.860379:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.860396:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888047480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534080768:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.860416:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.860435:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048864fbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534081280:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.860455:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.860483:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534079744 00000100:00100000:3.0:1789760766.860486:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534079744 00000100:00100000:3.0:1789760766.860507:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534080384 00000100:00100000:3.0:1789760766.860510:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534080384 00000100:00100000:3.0:1789760766.860533:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 947 00000100:00100000:3.0:1789760766.860539:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534079744:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.860584:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534079744:12345-0@lo:400:kworker.0 Request processed in 44us (249us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.860590:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 947 00000100:00100000:3.0:1789760766.860601:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 948 00000100:00100000:3.0:1789760766.860606:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534080384:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760766.861572:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534081152 00000100:00100000:2.0:1789760766.861572:0:8262:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534080640 00000100:00100000:0.0:1789760766.861582:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534081152 00000100:00100000:2.0:1789760766.861582:0:8262:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534080640 00000100:00100000:0.0:1789760766.861721:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534080768 00000100:00100000:0.0:1789760766.861725:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534080768 00000100:00100000:3.0:1789760766.861822:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534080384:12345-0@lo:400:kworker.0 Request processed in 1213us (1440us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.861834:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 948 00000100:00100000:1.0:1789760766.862387:0:8103:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534081280 00000100:00100000:1.0:1789760766.862393:0:8103:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534081280 00000100:00100000:3.0:1789760766.862443:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:3.0:1789760766.862453:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534081152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.862497:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534081152:12345-0@lo:400:kworker.0 Request processed in 46us (7908us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.862504:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00000100:00100000:2.0:1789760766.865234:0:8262:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 236 00000100:00100000:2.0:1789760766.865256:0:8262:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534080640:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.865351:0:8262:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534080640:12345-0@lo:400:kworker.0 Request processed in 106us (10804us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760766.865372:0:8102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1789760766.865384:0:8102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534080768:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760766.865420:0:8102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534080768:12345-0@lo:400:kworker.0 Request processed in 40us (5003us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760766.865427:0:8102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1789760766.865544:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888046a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3472:1876699534080512:0@lo:400:kworker.0 00000100:00100000:0.0:1789760766.865559:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888045180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3472:1876699534081024:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.872047:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534081536:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760766.872069:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534081664:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760766.872082:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888045c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3473:1876699534079616:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.872093:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888044380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534080128:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.872102:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888045500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534080640:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.872112:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888047800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534081152:0@lo:400:kworker.0 00000100:00100000:0.0:1789760766.872757:0:8102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:0.0:1789760766.872771:0:8102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534081280:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760766.872854:0:8102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534081280:12345-0@lo:400:kworker.0 Request processed in 88us (12398us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760766.872864:0:8102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00080000:1.0:1789760766.875459:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760766.875534:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888045880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534079744:0@lo:400:kworker.0 00000100:00100000:1.0:1789760766.875553:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888045f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534080384:0@lo:400:kworker.0 00000100:00100000:1.0:1789760766.875563:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888047480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534080768:0@lo:400:kworker.0 00000100:00100000:1.0:1789760766.875573:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048864fbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534081280:0@lo:400:kworker.0 00000100:00100000:2.0:1789760766.875776:0:8262:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 236 00002000:00010000:3.0:1789760766.887333:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23199744 -> 23203840 00010000:00010000:3.0:1789760766.887353:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe561ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.887367:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe561ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.887376:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe561ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.887383:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760766.887389:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe561ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.887398:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe561ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.887416:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe561ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.887424:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe561ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23199744->23203839] (req 23199744->23203839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.887557:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508409472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48549us (48793us total) trans 4294972984 rc 0/0 00000100:00100000:3.0:1789760766.887576:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5696 00000100:00100000:3.0:1789760766.887586:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745886401612 time 745943703073 state 0x2 ref 0 00000001:00080000:1.0:1789760766.887764:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972984 is committed 00000100:00100000:3.0:1789760766.892771:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760766.892862:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508409600 00000100:00100000:1.0:1789760766.892871:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508409600 00000100:00100000:1.0:1789760766.892932:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745886401612 time 745943703073 state 0x0 ref 1 00000100:00100000:1.0:1789760766.892948:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745943963489 time 745943703073 state 0x1 ref 1 00000100:00100000:1.0:1789760766.892976:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745943963489 time 745997686275 state 0x0 ref 1 00000100:00100000:1.0:1789760766.892988:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508409600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760766.893032:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23203840, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:1.0:1789760766.893041:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.893056:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=745s v=(0 0) 00010000:00010000:1.0:1789760766.893061:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.893067:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.893081:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760766.958148:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23203840 -> 23207936 00010000:00010000:1.0:1789760766.958168:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.958183:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.958191:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.958198:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760766.958203:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.958211:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.958242:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.958250:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23203840->23207935] (req 23203840->23207935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760766.958383:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508409600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 65390us (65616us total) trans 4294972985 rc 0/0 00000100:00100000:1.0:1789760766.958398:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5697 00000100:00100000:1.0:1789760766.958408:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745943963489 time 745997686275 state 0x2 ref 0 00000001:00080000:3.0:1789760766.959399:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972985 is committed 00000100:00100000:0.0:1789760766.964149:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760766.964252:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508409728 00000100:00100000:1.0:1789760766.964261:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508409728 00000100:00100000:1.0:1789760766.964345:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745943963489 time 745997686275 state 0x0 ref 1 00000100:00100000:1.0:1789760766.964361:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745997946691 time 745997686275 state 0x1 ref 1 00000100:00100000:1.0:1789760766.964390:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745997946691 time 746069101196 state 0x0 ref 1 00000100:00100000:1.0:1789760766.964404:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508409728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760766.964451:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23207936, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:1.0:1789760766.964460:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.964476:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760766.964481:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760766.964487:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760766.964501:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.010939:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972986 is committed 00002000:00010000:1.0:1789760767.011038:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23207936 -> 23212032 00010000:00010000:1.0:1789760767.011054:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.011067:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.011076:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.011082:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760767.011086:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.011094:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.011112:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.011119:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe561fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23207936->23212031] (req 23207936->23212031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.011251:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508409728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46845us (47106us total) trans 4294972986 rc 0/0 00000100:00100000:1.0:1789760767.011262:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5698 00000100:00100000:1.0:1789760767.011272:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745997946691 time 746069101196 state 0x2 ref 0 00000100:00100000:1.0:1789760767.021252:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760767.022382:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508409856 00000100:00100000:3.0:1789760767.022394:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508409856 00000100:00100000:3.0:1789760767.022591:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 745997946691 time 746069101196 state 0x0 ref 1 00000100:00100000:3.0:1789760767.022607:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746069361612 time 746069101196 state 0x1 ref 1 00000100:00100000:3.0:1789760767.022629:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746069361612 time 746127339908 state 0x0 ref 1 00000100:00100000:3.0:1789760767.022644:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508409856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760767.022687:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23212032, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:3.0:1789760767.022696:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56204 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.022712:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760767.022716:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.022721:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56204 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.022734:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56204 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760767.022935:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760767.045196:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972987 is committed 00002000:00010000:2.0:1789760767.045403:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23212032 -> 23216128 00010000:00010000:2.0:1789760767.045419:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56204 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.045432:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56204 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.045441:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56204 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.045454:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760767.045459:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56204 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.045469:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56204 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.045488:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56204 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.045495:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56204 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23212032->23216127] (req 23212032->23216127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.045597:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508409856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22949us (24349us total) trans 4294972987 rc 0/0 00000100:00100000:2.0:1789760767.045607:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5699 00000100:00100000:2.0:1789760767.045617:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746069361612 time 746127339908 state 0x2 ref 0 00000100:00100000:3.0:1789760767.055161:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760767.057979:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508409984 00000100:00100000:1.0:1789760767.057994:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508409984 00000100:00100000:1.0:1789760767.058060:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746069361612 time 746127339908 state 0x0 ref 1 00000100:00100000:1.0:1789760767.058076:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746127600324 time 746127339908 state 0x1 ref 1 00000100:00100000:1.0:1789760767.058113:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746127600324 time 746162823723 state 0x0 ref 1 00000100:00100000:1.0:1789760767.058126:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508409984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760767.058172:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23216128, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:1.0:1789760767.058181:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5620b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.058195:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760767.058199:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.058205:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5620b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.058219:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5620b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.120593:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972988 is committed 00002000:00010000:2.0:1789760767.120697:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23216128 -> 23220224 00010000:00010000:2.0:1789760767.120712:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5620b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.120724:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5620b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.120738:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5620b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.120745:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760767.120750:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5620b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.120759:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5620b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.120779:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5620b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.120787:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5620b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23216128->23220223] (req 23216128->23220223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.120880:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508409984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 62751us (65722us total) trans 4294972988 rc 0/0 00000100:00100000:2.0:1789760767.120899:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5700 00000100:00100000:2.0:1789760767.120914:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746127600324 time 746162823723 state 0x2 ref 0 00000100:00100000:3.0:1789760767.125701:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760767.126114:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508410112 00000100:00100000:1.0:1789760767.126123:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508410112 00000100:00100000:1.0:1789760767.126189:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746127600324 time 746162823723 state 0x0 ref 1 00000100:00100000:1.0:1789760767.126205:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746163084139 time 746162823723 state 0x1 ref 1 00000100:00100000:1.0:1789760767.126244:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746163084139 time 746230953605 state 0x0 ref 1 00000100:00100000:1.0:1789760767.126257:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508410112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760767.126340:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23220224, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:1.0:1789760767.126349:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56212 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.126377:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760767.126382:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.126388:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56212 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.126404:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56212 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.175904:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972989 is committed 00002000:00010000:3.0:1789760767.176063:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23220224 -> 23224320 00010000:00010000:3.0:1789760767.176078:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56212 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.176091:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56212 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.176101:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56212 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.176114:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760767.176118:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56212 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.176127:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56212 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.176148:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56212 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.176158:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56212 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23220224->23224319] (req 23220224->23224319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.176257:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508410112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49997us (50561us total) trans 4294972989 rc 0/0 00000100:00100000:3.0:1789760767.176267:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5701 00000100:00100000:3.0:1789760767.176277:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746163084139 time 746230953605 state 0x2 ref 0 00000100:00100000:2.0:1789760767.182646:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760767.183221:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508410240 00000100:00100000:3.0:1789760767.183232:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508410240 00000100:00100000:3.0:1789760767.183337:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746163084139 time 746230953605 state 0x0 ref 1 00000100:00100000:3.0:1789760767.183352:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746231214021 time 746230953605 state 0x1 ref 1 00000100:00100000:3.0:1789760767.183392:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746231214021 time 746288102429 state 0x0 ref 1 00000100:00100000:3.0:1789760767.183409:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508410240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760767.183460:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23224320, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:3.0:1789760767.183469:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56219 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.183483:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760767.183487:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.183494:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56219 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.183509:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56219 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.222789:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972990 is committed 00002000:00010000:3.0:1789760767.223041:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23224320 -> 23228416 00010000:00010000:3.0:1789760767.223056:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56219 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.223069:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56219 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.223079:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56219 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.223087:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760767.223092:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56219 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.223102:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56219 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.223122:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56219 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.223130:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56219 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23224320->23228415] (req 23224320->23228415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.223229:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508410240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39818us (40588us total) trans 4294972990 rc 0/0 00000100:00100000:3.0:1789760767.223249:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5702 00000100:00100000:3.0:1789760767.223258:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746231214021 time 746288102429 state 0x2 ref 0 00000100:00100000:0.0:1789760767.235400:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760767.235569:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508410368 00000100:00100000:2.0:1789760767.235579:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508410368 00000100:00100000:2.0:1789760767.235632:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746231214021 time 746288102429 state 0x0 ref 1 00000100:00100000:2.0:1789760767.235652:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746288362845 time 746288102429 state 0x1 ref 1 00000100:00100000:2.0:1789760767.235722:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746288362845 time 746340432356 state 0x0 ref 1 00000100:00100000:2.0:1789760767.235735:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508410368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760767.235788:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23228416, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:2.0:1789760767.235798:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56220 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.235826:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760767.235830:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.235837:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56220 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.235852:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56220 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.263149:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972991 is committed 00002000:00010000:3.0:1789760767.263236:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23228416 -> 23232512 00010000:00010000:3.0:1789760767.263252:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56220 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.263264:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56220 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.263272:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56220 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.263486:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760767.263491:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56220 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.263502:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56220 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.263522:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56220 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.263530:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56220 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23228416->23232511] (req 23228416->23232511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.263660:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508410368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27922us (28266us total) trans 4294972991 rc 0/0 00000100:00100000:3.0:1789760767.263671:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5703 00000100:00100000:3.0:1789760767.263681:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746288362845 time 746340432356 state 0x2 ref 0 00000100:00100000:0.0:1789760767.275092:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760767.275178:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508410496 00000100:00100000:3.0:1789760767.275189:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508410496 00000100:00100000:3.0:1789760767.275251:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746288362845 time 746340432356 state 0x0 ref 1 00000100:00100000:3.0:1789760767.275266:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746340692772 time 746340432356 state 0x1 ref 1 00000100:00100000:3.0:1789760767.275329:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746340692772 time 746380039162 state 0x0 ref 1 00000100:00100000:3.0:1789760767.275342:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508410496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760767.275390:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23232512, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:3.0:1789760767.275400:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56227 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.275415:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760767.275419:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.275425:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56227 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.275695:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56227 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760767.320869:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972992 is committed 00002000:00010000:0.0:1789760767.320977:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23232512 -> 23236608 00010000:00010000:0.0:1789760767.320992:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56227 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.321004:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56227 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.321013:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56227 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.321020:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760767.321024:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56227 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.321033:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56227 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.321053:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56227 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.321066:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56227 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23232512->23236607] (req 23232512->23236607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.321160:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508410496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45815us (46073us total) trans 4294972992 rc 0/0 00000100:00100000:0.0:1789760767.321171:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5704 00000100:00100000:0.0:1789760767.323262:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746340692772 time 746380039162 state 0x2 ref 0 00000100:00100000:2.0:1789760767.327792:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760767.327876:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508410624 00000100:00100000:3.0:1789760767.327897:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508410624 00000100:00100000:3.0:1789760767.327949:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746340692772 time 746380039162 state 0x0 ref 1 00000100:00100000:3.0:1789760767.327964:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746380299578 time 746380039162 state 0x1 ref 1 00000100:00100000:3.0:1789760767.327992:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746380299578 time 746432703062 state 0x0 ref 1 00000100:00100000:3.0:1789760767.328005:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508410624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760767.328054:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23236608, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:3.0:1789760767.328063:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5622e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.328078:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760767.328082:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.328088:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5622e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.328103:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5622e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.357001:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972993 is committed 00002000:00010000:2.0:1789760767.357105:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23236608 -> 23240704 00010000:00010000:2.0:1789760767.357120:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5622e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.357133:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5622e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.357147:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5622e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.357155:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760767.357161:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5622e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.357170:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5622e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.357190:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5622e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.357199:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5622e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23236608->23240703] (req 23236608->23240703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.357334:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508410624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29326us (29548us total) trans 4294972993 rc 0/0 00000100:00100000:2.0:1789760767.357566:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5705 00000100:00100000:2.0:1789760767.357577:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746380299578 time 746432703062 state 0x2 ref 0 00000100:00100000:3.0:1789760767.367508:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760767.367579:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508410752 00000100:00100000:2.0:1789760767.367589:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508410752 00000100:00100000:2.0:1789760767.367644:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746380299578 time 746432703062 state 0x0 ref 1 00000100:00100000:2.0:1789760767.367659:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746432963478 time 746432703062 state 0x1 ref 1 00000100:00100000:2.0:1789760767.367694:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746432963478 time 746472404136 state 0x0 ref 1 00000100:00100000:2.0:1789760767.367707:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508410752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760767.367758:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23240704, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:2.0:1789760767.367767:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56235 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.367783:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760767.367787:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.367793:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56235 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.367805:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56235 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760767.422619:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23240704 -> 23244800 00010000:00010000:2.0:1789760767.422640:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56235 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.422654:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56235 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.422664:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56235 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.422672:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760767.422677:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56235 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.422687:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56235 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.422707:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56235 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.422715:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56235 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23240704->23244799] (req 23240704->23244799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760767.422811:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508410752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55102us (55408us total) trans 4294972994 rc 0/0 00000100:00100000:2.0:1789760767.422822:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5706 00000100:00100000:2.0:1789760767.422832:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746432963478 time 746472404136 state 0x2 ref 0 00000001:00080000:3.0:1789760767.423596:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972994 is committed 00000100:00100000:3.0:1789760767.428666:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760767.428828:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508410880 00000100:00100000:0.0:1789760767.428838:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508410880 00000100:00100000:0.0:1789760767.428901:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746432963478 time 746472404136 state 0x0 ref 1 00000100:00100000:0.0:1789760767.428916:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746472664552 time 746472404136 state 0x1 ref 1 00000100:00100000:0.0:1789760767.428949:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746472664552 time 746533659202 state 0x0 ref 1 00000100:00100000:0.0:1789760767.428963:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508410880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760767.429009:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23244800, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:0.0:1789760767.429018:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5623c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.429033:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760767.429038:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.429044:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5623c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.429064:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5623c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.477218:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972995 is committed 00002000:00010000:0.0:1789760767.477442:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23244800 -> 23248896 00010000:00010000:0.0:1789760767.477457:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5623c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.477468:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5623c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.477477:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5623c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.477485:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760767.477489:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5623c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.477498:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5623c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.477517:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5623c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.477524:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5623c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23244800->23248895] (req 23244800->23248895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.477718:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508410880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48752us (49057us total) trans 4294972995 rc 0/0 00000100:00100000:0.0:1789760767.477729:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5707 00000100:00100000:0.0:1789760767.477739:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746472664552 time 746533659202 state 0x2 ref 0 00000100:00100000:1.0:1789760767.485905:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760767.486026:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508411008 00000100:00100000:2.0:1789760767.486036:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508411008 00000100:00100000:2.0:1789760767.486092:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746472664552 time 746533659202 state 0x0 ref 1 00000100:00100000:2.0:1789760767.486107:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746533919618 time 746533659202 state 0x1 ref 1 00000100:00100000:2.0:1789760767.486170:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746533919618 time 746590879801 state 0x0 ref 1 00000100:00100000:2.0:1789760767.486183:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508411008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760767.486240:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23248896, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0xb807 00010000:00010000:2.0:1789760767.486250:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56243 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.486265:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760767.486270:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.486276:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56243 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.486319:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56243 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.526480:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23248896 -> 23252992 00002000:00010000:0.0:1789760767.526489:0:14638:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760767 -> 1789760768 00002000:00010000:0.0:1789760767.526491:0:14638:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760767 -> 1789760768 00010000:00010000:0.0:1789760767.526504:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56243 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.526516:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56243 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.526526:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56243 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.526533:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760767.526538:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56243 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.526549:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56243 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.526570:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56243 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.526579:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56243 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23248896->23252991] (req 23248896->23252991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.526745:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508411008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40559us (40859us total) trans 4294972996 rc 0/0 00000100:00100000:0.0:1789760767.526756:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5708 00000100:00100000:0.0:1789760767.526766:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746533919618 time 746590879801 state 0x2 ref 0 00000001:00080000:1.0:1789760767.526925:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972996 is committed 00000100:00100000:3.0:1789760767.531752:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760767.531843:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508411136 00000100:00100000:2.0:1789760767.531853:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508411136 00000100:00100000:2.0:1789760767.531916:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746533919618 time 746590879801 state 0x0 ref 1 00000100:00100000:2.0:1789760767.531932:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746591140217 time 746590879801 state 0x1 ref 1 00000100:00100000:2.0:1789760767.532001:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746591140217 time 746636710647 state 0x0 ref 1 00000100:00100000:2.0:1789760767.532014:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508411136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760767.532064:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23252992, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:2.0:1789760767.532073:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5624a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.532089:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760767.532094:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.532100:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5624a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.532116:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5624a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.567276:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23252992 -> 23257088 00010000:00010000:3.0:1789760767.567322:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5624a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.567335:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5624a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.567344:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5624a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.567351:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760767.567356:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5624a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.567365:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5624a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.567390:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5624a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.567399:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5624a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23252992->23257087] (req 23252992->23257087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.567486:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508411136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35470us (35739us total) trans 4294972997 rc 0/0 00000100:00100000:3.0:1789760767.567497:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5709 00000100:00100000:3.0:1789760767.567507:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746591140217 time 746636710647 state 0x2 ref 0 00000001:00080000:0.0:1789760767.568029:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972997 is committed 00000100:00100000:3.0:1789760767.575243:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760767.575718:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508411264 00000100:00100000:0.0:1789760767.575730:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508411264 00000100:00100000:0.0:1789760767.575784:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746591140217 time 746636710647 state 0x0 ref 1 00000100:00100000:0.0:1789760767.575799:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746636971063 time 746636710647 state 0x1 ref 1 00000100:00100000:0.0:1789760767.575834:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746636971063 time 746680544402 state 0x0 ref 1 00000100:00100000:0.0:1789760767.575846:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508411264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760767.575907:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23257088, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:0.0:1789760767.575916:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56251 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.575930:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:0.0:1789760767.575934:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.575940:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56251 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.575953:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56251 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760767.649796:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972998 is committed 00002000:00010000:0.0:1789760767.651435:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23257088 -> 23261184 00010000:00010000:0.0:1789760767.651451:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56251 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.651464:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56251 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.651472:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56251 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.651480:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760767.651485:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56251 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.651495:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56251 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.651516:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56251 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.651525:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56251 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23257088->23261183] (req 23257088->23261183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.651683:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508411264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 75834us (76445us total) trans 4294972998 rc 0/0 00000100:00100000:0.0:1789760767.651694:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5710 00000100:00100000:0.0:1789760767.651704:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746636971063 time 746680544402 state 0x2 ref 0 00000100:00100000:2.0:1789760767.658582:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760767.660403:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508411392 00000100:00100000:0.0:1789760767.660416:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508411392 00000100:00100000:2.0:1789760767.664472:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746636971063 time 746680544402 state 0x0 ref 1 00000100:00100000:2.0:1789760767.664491:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746680804818 time 746680544402 state 0x1 ref 1 00000100:00100000:2.0:1789760767.664591:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746680804818 time 746769300028 state 0x0 ref 1 00000100:00100000:2.0:1789760767.664608:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508411392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760767.664658:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23261184, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:2.0:1789760767.664666:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56258 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.664683:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760767.664687:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.664695:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56258 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.664709:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56258 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.702126:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23261184 -> 23265280 00010000:00010000:3.0:1789760767.702146:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56258 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.702159:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56258 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.702168:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56258 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.702175:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760767.702180:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56258 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.702195:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56258 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.702214:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56258 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.702222:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56258 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23261184->23265279] (req 23261184->23265279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.702414:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508411392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37804us (43835us total) trans 4294972999 rc 0/0 00000100:00100000:3.0:1789760767.702425:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5711 00000100:00100000:3.0:1789760767.702434:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746680804818 time 746769300028 state 0x2 ref 0 00000001:00080000:1.0:1789760767.705199:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294972999 is committed 00000100:00100000:0.0:1789760767.706761:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760767.706928:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508411520 00000100:00100000:3.0:1789760767.706938:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508411520 00000100:00100000:3.0:1789760767.706991:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746680804818 time 746769300028 state 0x0 ref 1 00000100:00100000:3.0:1789760767.707005:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746769560444 time 746769300028 state 0x1 ref 1 00000100:00100000:3.0:1789760767.707072:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746769560444 time 746811780877 state 0x0 ref 1 00000100:00100000:3.0:1789760767.707085:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508411520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760767.707132:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23265280, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:3.0:1789760767.707141:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5625f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.707155:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:3.0:1789760767.707159:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.707165:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5625f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.707178:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5625f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760767.739630:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973000 is committed 00002000:00010000:1.0:1789760767.739737:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23265280 -> 23269376 00010000:00010000:1.0:1789760767.739752:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5625f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.739765:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5625f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.739781:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5625f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.739800:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760767.739806:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5625f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.739815:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5625f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.739834:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5625f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.739842:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5625f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23265280->23269375] (req 23265280->23269375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.739947:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508411520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32859us (33189us total) trans 4294973000 rc 0/0 00000100:00100000:1.0:1789760767.739957:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5712 00000100:00100000:1.0:1789760767.739967:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746769560444 time 746811780877 state 0x2 ref 0 00010000:00010000:3.0:1789760767.742407:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760767.744714:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760767.744859:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508411648 00000100:00100000:2.0:1789760767.744869:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508411648 00000100:00100000:2.0:1789760767.744930:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746769560444 time 746811780877 state 0x0 ref 1 00000100:00100000:2.0:1789760767.744945:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746812041293 time 746811780877 state 0x1 ref 1 00000100:00100000:2.0:1789760767.745022:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746812041293 time 746849731489 state 0x0 ref 1 00000100:00100000:2.0:1789760767.745035:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508411648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760767.745083:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23269376, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:2.0:1789760767.745092:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56266 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.745105:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:2.0:1789760767.745110:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760767.745116:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56266 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760767.745130:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56266 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760767.844923:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23269376 -> 23273472 00010000:00010000:0.0:1789760767.844942:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56266 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.844955:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56266 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.844963:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56266 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.844970:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760767.844975:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56266 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.844990:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56266 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.845019:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56266 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.845027:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56266 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23269376->23273471] (req 23269376->23273471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760767.845414:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508411648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 100374us (100702us total) trans 4294973001 rc 0/0 00000100:00100000:0.0:1789760767.845426:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5713 00000100:00100000:0.0:1789760767.845435:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746812041293 time 746849731489 state 0x2 ref 0 00000001:00080000:0.0:1789760767.845473:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973001 is committed 00000100:00100000:3.0:1789760767.849828:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760767.850047:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508411776 00000100:00100000:1.0:1789760767.850056:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508411776 00000100:00100000:1.0:1789760767.850108:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746812041293 time 746849731489 state 0x0 ref 1 00000100:00100000:1.0:1789760767.850123:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746849991905 time 746849731489 state 0x1 ref 1 00000100:00100000:1.0:1789760767.850157:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746849991905 time 746954866872 state 0x0 ref 1 00000100:00100000:1.0:1789760767.850169:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508411776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760767.850217:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23273472, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:1.0:1789760767.850238:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5626d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.850254:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=746s v=(0 0) 00010000:00010000:1.0:1789760767.850258:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.850265:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5626d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.850277:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5626d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.871438:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760767.871461:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b66e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534081792:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760767.871533:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760767.871605:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534081792 00000100:00100000:3.0:1789760767.871612:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534081792 00000100:00100000:3.0:1789760767.871656:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 452 00000100:00100000:3.0:1789760767.871665:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534081792:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760767.871702:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789760767.871746:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534081792:12345-0@lo:41:osp-pre-0-1.0 Request processed in 82us (213us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760767.871753:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 452 00000100:00100000:3.0:1789760767.872328:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b66e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534081792:0@lo:41:osp-pre-0-1.0 00000001:00080000:0.0:1789760767.902006:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973002 is committed 00002000:00010000:1.0:1789760767.902170:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23273472 -> 23277568 00010000:00010000:1.0:1789760767.902185:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5626d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.902198:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5626d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.902207:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5626d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.902216:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760767.902221:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5626d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.902242:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5626d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.902265:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5626d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.902273:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5626d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23273472->23277567] (req 23273472->23277567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.902392:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508411776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52219us (52568us total) trans 4294973002 rc 0/0 00000100:00100000:1.0:1789760767.902402:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5714 00000100:00100000:1.0:1789760767.902411:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746849991905 time 746954866872 state 0x2 ref 0 00000100:00100000:0.0:1789760767.910161:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760767.911055:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508411904 00000100:00100000:1.0:1789760767.911066:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508411904 00000100:00100000:1.0:1789760767.911121:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746849991905 time 746954866872 state 0x0 ref 1 00000100:00100000:1.0:1789760767.911137:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746955127288 time 746954866872 state 0x1 ref 1 00000100:00100000:1.0:1789760767.911166:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746955127288 time 747015877315 state 0x0 ref 1 00000100:00100000:1.0:1789760767.911179:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508411904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760767.911394:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23277568, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:1.0:1789760767.911406:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56274 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.911422:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760767.911427:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760767.911433:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56274 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.911448:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56274 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760767.958942:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23277568 -> 23281664 00010000:00010000:1.0:1789760767.958961:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56274 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.958975:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56274 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.958984:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56274 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.958991:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760767.958996:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56274 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.959006:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.959030:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56274 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760767.959037:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56274 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23277568->23281663] (req 23277568->23281663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760767.959125:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508411904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47943us (48966us total) trans 4294973003 rc 0/0 00000100:00100000:1.0:1789760767.959135:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5715 00000100:00100000:1.0:1789760767.959144:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746955127288 time 747015877315 state 0x2 ref 0 00000001:00080000:0.0:1789760767.959840:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973003 is committed 00000100:00100000:0.0:1789760767.967537:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760767.968134:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508412032 00000100:00100000:1.0:1789760767.968144:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508412032 00000100:00100000:1.0:1789760767.968586:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 746955127288 time 747015877315 state 0x0 ref 1 00000100:00100000:1.0:1789760767.968604:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747016137731 time 747015877315 state 0x1 ref 1 00000100:00100000:0.0:1789760767.968747:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747016137731 time 747073455083 state 0x0 ref 1 00000100:00100000:0.0:1789760767.968763:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508412032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760767.968825:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23281664, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:0.0:1789760767.968835:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5627b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.968852:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760767.968856:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760767.968863:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5627b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760767.968887:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5627b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.006120:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973004 is committed 00002000:00010000:0.0:1789760768.006313:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23281664 -> 23285760 00010000:00010000:0.0:1789760768.006330:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5627b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.006344:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5627b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.006353:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5627b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.006361:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760768.006377:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5627b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.006387:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5627b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.006409:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5627b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.006418:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5627b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23281664->23285759] (req 23281664->23285759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.006507:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508412032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37744us (38974us total) trans 4294973004 rc 0/0 00000100:00100000:0.0:1789760768.006518:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5716 00000100:00100000:0.0:1789760768.006528:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747016137731 time 747073455083 state 0x2 ref 0 00000100:00100000:1.0:1789760768.017329:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760768.017421:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508412160 00000100:00100000:3.0:1789760768.017431:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508412160 00000100:00100000:3.0:1789760768.017490:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747016137731 time 747073455083 state 0x0 ref 1 00000100:00100000:3.0:1789760768.017508:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747073715499 time 747073455083 state 0x1 ref 1 00000100:00100000:3.0:1789760768.017530:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747073715499 time 747122241045 state 0x0 ref 1 00000100:00100000:3.0:1789760768.017542:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508412160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760768.017594:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23285760, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:3.0:1789760768.017603:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56282 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.017620:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760768.017625:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.017632:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56282 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.017647:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56282 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.077324:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23285760 -> 23289856 00010000:00010000:3.0:1789760768.077345:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56282 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.077358:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56282 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.077367:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56282 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.077375:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760768.077380:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56282 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.077390:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56282 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.077409:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56282 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.077417:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56282 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23285760->23289855] (req 23285760->23289855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.077510:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508412160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59966us (60227us total) trans 4294973005 rc 0/0 00000100:00100000:3.0:1789760768.077521:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5717 00000100:00100000:3.0:1789760768.077532:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747073715499 time 747122241045 state 0x2 ref 0 00000001:00080000:0.0:1789760768.078016:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973005 is committed 00000100:00100000:0.0:1789760768.086054:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760768.089334:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508412288 00000100:00100000:1.0:1789760768.089351:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508412288 00000100:00100000:1.0:1789760768.089420:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747073715499 time 747122241045 state 0x0 ref 1 00000100:00100000:1.0:1789760768.089437:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747122501461 time 747122241045 state 0x1 ref 1 00000100:00100000:1.0:1789760768.089475:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747122501461 time 747194184477 state 0x0 ref 1 00000100:00100000:1.0:1789760768.089491:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508412288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760768.089541:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23289856, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:1.0:1789760768.089552:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56289 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.089569:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760768.089574:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.089580:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56289 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.089594:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56289 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.146142:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973006 is committed 00002000:00010000:1.0:1789760768.146654:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23289856 -> 23293952 00010000:00010000:1.0:1789760768.146669:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56289 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.146682:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56289 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.146692:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56289 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.146699:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760768.146704:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56289 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.146713:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56289 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.146733:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56289 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.146741:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56289 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23289856->23293951] (req 23289856->23293951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.147052:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508412288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57560us (61002us total) trans 4294973006 rc 0/0 00000100:00100000:1.0:1789760768.147064:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5718 00000100:00100000:1.0:1789760768.147074:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747122501461 time 747194184477 state 0x2 ref 0 00000100:00100000:1.0:1789760768.157158:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760768.157261:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508412416 00000100:00100000:0.0:1789760768.157270:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508412416 00000100:00100000:0.0:1789760768.157368:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747122501461 time 747194184477 state 0x0 ref 1 00000100:00100000:0.0:1789760768.157385:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747194444893 time 747194184477 state 0x1 ref 1 00000100:00100000:0.0:1789760768.157417:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747194444893 time 747262127392 state 0x0 ref 1 00000100:00100000:0.0:1789760768.157430:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508412416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760768.157475:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23293952, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:0.0:1789760768.157484:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56290 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.157499:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760768.157504:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.157510:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56290 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.157530:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56290 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.194682:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973007 is committed 00002000:00010000:0.0:1789760768.194868:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23293952 -> 23298048 00010000:00010000:0.0:1789760768.194894:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56290 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.194911:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56290 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.194920:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56290 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.194927:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760768.194933:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56290 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.194942:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56290 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.194961:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56290 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.194974:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56290 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23293952->23298047] (req 23293952->23298047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.195069:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508412416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37636us (37922us total) trans 4294973007 rc 0/0 00000100:00100000:0.0:1789760768.195080:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5719 00000100:00100000:0.0:1789760768.195089:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747194444893 time 747262127392 state 0x2 ref 0 00000100:00100000:2.0:1789760768.199126:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760768.199886:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508412544 00000100:00100000:3.0:1789760768.199899:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508412544 00000100:00100000:2.0:1789760768.200046:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747194444893 time 747262127392 state 0x0 ref 1 00000100:00100000:2.0:1789760768.200062:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747262387808 time 747262127392 state 0x1 ref 1 00000100:00100000:2.0:1789760768.200133:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747262387808 time 747304842687 state 0x0 ref 1 00000100:00100000:2.0:1789760768.200311:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508412544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760768.200428:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23298048, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:2.0:1789760768.200437:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56297 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.200454:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760768.200458:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.200465:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56297 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.200480:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56297 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760768.255114:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23298048 -> 23302144 00010000:00010000:2.0:1789760768.255134:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56297 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.255159:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56297 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.255176:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56297 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.255184:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760768.255189:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56297 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.255200:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56297 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.255219:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56297 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.255516:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56297 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23298048->23302143] (req 23298048->23302143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.255624:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508412544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55402us (56504us total) trans 4294973008 rc 0/0 00000100:00100000:2.0:1789760768.255635:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5720 00000100:00100000:2.0:1789760768.255645:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747262387808 time 747304842687 state 0x2 ref 0 00000001:00080000:0.0:1789760768.255725:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973008 is committed 00000100:00100000:2.0:1789760768.260607:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760768.260658:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508412672 00000100:00100000:3.0:1789760768.260667:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508412672 00000100:00100000:3.0:1789760768.260722:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747262387808 time 747304842687 state 0x0 ref 1 00000100:00100000:3.0:1789760768.260737:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747305103103 time 747304842687 state 0x1 ref 1 00000100:00100000:3.0:1789760768.260868:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747305103103 time 747365577418 state 0x0 ref 1 00000100:00100000:3.0:1789760768.260893:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508412672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760768.260936:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23302144, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:3.0:1789760768.260943:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5629e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.260957:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:3.0:1789760768.260962:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.260968:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5629e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.260983:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5629e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.301725:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973009 is committed 00002000:00010000:0.0:1789760768.301904:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23302144 -> 23306240 00010000:00010000:0.0:1789760768.301922:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5629e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.301934:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5629e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.301944:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5629e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.301954:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760768.301958:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5629e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.301967:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5629e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.301985:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5629e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.301998:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe5629e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23302144->23306239] (req 23302144->23306239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.302095:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508412672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41200us (41491us total) trans 4294973009 rc 0/0 00000100:00100000:0.0:1789760768.302104:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5721 00000100:00100000:0.0:1789760768.302115:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747305103103 time 747365577418 state 0x2 ref 0 00000100:00100000:3.0:1789760768.305557:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760768.305749:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508412800 00000100:00100000:0.0:1789760768.305759:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508412800 00000100:00100000:0.0:1789760768.305823:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747305103103 time 747365577418 state 0x0 ref 1 00000100:00100000:0.0:1789760768.305839:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747365837834 time 747365577418 state 0x1 ref 1 00000100:00100000:0.0:1789760768.305863:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747365837834 time 747410573794 state 0x0 ref 1 00000100:00100000:0.0:1789760768.305889:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508412800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760768.305940:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23306240, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:0.0:1789760768.305949:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.305965:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760768.305969:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.305976:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.305996:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.340484:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23306240 -> 23310336 00010000:00010000:0.0:1789760768.340504:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.340518:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.340528:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.340536:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760768.340540:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.340549:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.340569:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.340577:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23306240->23310335] (req 23306240->23310335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.340677:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508412800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34798us (35124us total) trans 4294973010 rc 0/0 00000100:00100000:0.0:1789760768.340688:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5722 00000100:00100000:0.0:1789760768.340697:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747365837834 time 747410573794 state 0x2 ref 0 00000001:00080000:3.0:1789760768.345266:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973010 is committed 00000100:00100000:2.0:1789760768.345637:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760768.345708:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508412928 00000100:00100000:1.0:1789760768.345717:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508412928 00000100:00100000:1.0:1789760768.345764:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747365837834 time 747410573794 state 0x0 ref 1 00000100:00100000:1.0:1789760768.345779:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747410834210 time 747410573794 state 0x1 ref 1 00000100:00100000:1.0:1789760768.345895:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747410834210 time 747450604879 state 0x0 ref 1 00000100:00100000:1.0:1789760768.345908:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508412928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760768.345951:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23310336, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:1.0:1789760768.345960:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe562ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.345981:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760768.345985:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.345992:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe562ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.346005:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe562ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760768.446050:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23310336 -> 23314432 00010000:00010000:2.0:1789760768.446072:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe562ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.446086:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe562ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.446096:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe562ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.446105:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760768.446109:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe562ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.446118:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe562ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.446140:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe562ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.446149:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe562ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23310336->23314431] (req 23310336->23314431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.446279:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508412928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 100358us (100632us total) trans 4294973011 rc 0/0 00000100:00100000:2.0:1789760768.446346:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5723 00000100:00100000:2.0:1789760768.446357:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747410834210 time 747450604879 state 0x2 ref 0 00000001:00080000:3.0:1789760768.446991:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973011 is committed 00000100:00100000:2.0:1789760768.451098:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760768.451154:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508413056 00000100:00100000:0.0:1789760768.451164:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508413056 00000100:00100000:0.0:1789760768.451216:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747410834210 time 747450604879 state 0x0 ref 1 00000100:00100000:0.0:1789760768.451318:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747450865295 time 747450604879 state 0x1 ref 1 00000100:00100000:0.0:1789760768.451498:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747450865295 time 747556206528 state 0x0 ref 1 00000100:00100000:0.0:1789760768.451514:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508413056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760768.451564:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23314432, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:0.0:1789760768.451574:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.451588:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760768.451593:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.451598:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.451612:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.486352:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23314432 -> 23318528 00000001:00080000:1.0:1789760768.486451:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973012 is committed 00010000:00010000:0.0:1789760768.486559:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.486573:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.486582:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.486590:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760768.486595:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.486604:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.486624:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.486632:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe562b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23314432->23318527] (req 23314432->23318527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.486709:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508413056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35195us (35617us total) trans 4294973012 rc 0/0 00000100:00100000:0.0:1789760768.486724:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5724 00000100:00100000:0.0:1789760768.486733:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747450865295 time 747556206528 state 0x2 ref 0 00000100:00100000:1.0:1789760768.491162:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760768.491261:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508413184 00000100:00100000:0.0:1789760768.491271:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508413184 00000100:00100000:0.0:1789760768.491358:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747450865295 time 747556206528 state 0x0 ref 1 00000100:00100000:0.0:1789760768.491373:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747556466944 time 747556206528 state 0x1 ref 1 00000100:00100000:0.0:1789760768.491459:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747556466944 time 747596168864 state 0x0 ref 1 00000100:00100000:0.0:1789760768.491472:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508413184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760768.491522:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23318528, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0xb807 00010000:00010000:0.0:1789760768.491531:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.491547:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760768.491552:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.491558:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.491572:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760768.542681:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973013 is committed 00002000:00010000:0.0:1789760768.542785:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23318528 -> 23322624 00002000:00010000:0.0:1789760768.542791:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760768 -> 1789760769 00002000:00010000:0.0:1789760768.542794:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760768 -> 1789760769 00010000:00010000:0.0:1789760768.542807:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.542820:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.542829:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.542837:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760768.542842:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.542851:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.542878:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.542886:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23318528->23322623] (req 23318528->23322623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.542981:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508413184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51506us (51824us total) trans 4294973013 rc 0/0 00000100:00100000:0.0:1789760768.542992:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5725 00000100:00100000:0.0:1789760768.543003:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747556466944 time 747596168864 state 0x2 ref 0 00000100:00100000:2.0:1789760768.548731:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760768.549126:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508413312 00000100:00100000:1.0:1789760768.549137:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508413312 00000100:00100000:1.0:1789760768.549205:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747556466944 time 747596168864 state 0x0 ref 1 00000100:00100000:1.0:1789760768.549221:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747596429280 time 747596168864 state 0x1 ref 1 00000100:00100000:1.0:1789760768.549320:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747596429280 time 747654028629 state 0x0 ref 1 00000100:00100000:1.0:1789760768.549333:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508413312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760768.549382:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23322624, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:1.0:1789760768.549400:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe562c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.549415:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760768.549421:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.549435:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe562c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.549449:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe562c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.602812:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973014 is committed 00002000:00010000:1.0:1789760768.602900:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23322624 -> 23326720 00010000:00010000:1.0:1789760768.602914:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe562c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.602926:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe562c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.602935:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe562c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.602942:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760768.602947:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe562c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.602956:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe562c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.602976:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe562c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.602988:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe562c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23322624->23326719] (req 23322624->23326719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.603184:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508413312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53848us (54463us total) trans 4294973014 rc 0/0 00000100:00100000:1.0:1789760768.603196:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5726 00000100:00100000:1.0:1789760768.603206:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747596429280 time 747654028629 state 0x2 ref 0 00000100:00100000:1.0:1789760768.608719:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760768.608888:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508413440 00000100:00100000:0.0:1789760768.608898:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508413440 00000100:00100000:0.0:1789760768.609017:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747596429280 time 747654028629 state 0x0 ref 1 00000100:00100000:0.0:1789760768.609033:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747654289045 time 747654028629 state 0x1 ref 1 00000100:00100000:0.0:1789760768.609064:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747654289045 time 747713774951 state 0x0 ref 1 00000100:00100000:0.0:1789760768.609077:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508413440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760768.609123:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23326720, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:0.0:1789760768.609132:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.609151:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760768.609156:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.609162:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.609175:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.662200:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973015 is committed 00002000:00010000:0.0:1789760768.662490:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23326720 -> 23330816 00010000:00010000:0.0:1789760768.662510:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.662526:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.662536:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.662547:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760768.662553:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.662562:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.662585:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.662596:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23326720->23330815] (req 23326720->23330815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.662702:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508413440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53623us (53987us total) trans 4294973015 rc 0/0 00000100:00100000:0.0:1789760768.662716:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5727 00000100:00100000:0.0:1789760768.662725:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747654289045 time 747713774951 state 0x2 ref 0 00000100:00100000:3.0:1789760768.667790:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760768.667892:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508413568 00000100:00100000:0.0:1789760768.667903:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508413568 00000100:00100000:0.0:1789760768.668016:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747654289045 time 747713774951 state 0x0 ref 1 00000100:00100000:0.0:1789760768.668029:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747714035367 time 747713774951 state 0x1 ref 1 00000100:00100000:0.0:1789760768.668052:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747714035367 time 747772764085 state 0x0 ref 1 00000100:00100000:0.0:1789760768.668061:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d40450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508413568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760768.668108:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23330816, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:0.0:1789760768.668117:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.668138:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760768.668142:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.668148:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.668171:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.713408:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23330816 -> 23334912 00010000:00010000:0.0:1789760768.713531:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.713545:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.713554:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.713562:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760768.713568:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.713577:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.713598:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.713617:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23330816->23334911] (req 23330816->23334911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.713714:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d40450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508413568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45648us (45938us total) trans 4294973016 rc 0/0 00000100:00100000:0.0:1789760768.713725:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5728 00000100:00100000:0.0:1789760768.713735:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747714035367 time 747772764085 state 0x2 ref 0 00000001:00080000:3.0:1789760768.714412:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973016 is committed 00000100:00100000:2.0:1789760768.719782:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760768.720924:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508413696 00000100:00100000:1.0:1789760768.720939:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508413696 00000100:00100000:1.0:1789760768.721189:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747714035367 time 747772764085 state 0x0 ref 1 00000100:00100000:1.0:1789760768.721206:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747773024501 time 747772764085 state 0x1 ref 1 00000100:00100000:1.0:1789760768.721299:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747773024501 time 747825993043 state 0x0 ref 1 00000100:00100000:1.0:1789760768.721314:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508413696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760768.721365:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23334912, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:1.0:1789760768.721374:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.721391:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760768.721395:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.721401:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.721416:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.761158:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973017 is committed 00002000:00010000:1.0:1789760768.761644:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23334912 -> 23339008 00010000:00010000:1.0:1789760768.761659:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.761671:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.761679:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.761686:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760768.761691:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.761700:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.761720:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.761728:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23334912->23339007] (req 23334912->23339007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.761877:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508413696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40552us (42102us total) trans 4294973017 rc 0/0 00000100:00100000:1.0:1789760768.761888:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5729 00000100:00100000:1.0:1789760768.761898:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747773024501 time 747825993043 state 0x2 ref 0 00010000:00010000:3.0:1789760768.766723:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760768.778743:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760768.778996:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508413824 00000100:00100000:0.0:1789760768.779008:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508413824 00000100:00100000:0.0:1789760768.779062:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747773024501 time 747825993043 state 0x0 ref 1 00000100:00100000:0.0:1789760768.779077:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747826253459 time 747825993043 state 0x1 ref 1 00000100:00100000:0.0:1789760768.779109:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747826253459 time 747883818679 state 0x0 ref 1 00000100:00100000:0.0:1789760768.779122:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508413824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760768.779181:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23339008, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:0.0:1789760768.779191:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.779207:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:0.0:1789760768.779211:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.779217:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.779242:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760768.824765:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23339008 -> 23343104 00010000:00010000:1.0:1789760768.824789:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.824801:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.824811:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.824818:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760768.824822:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.824832:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.824853:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.824875:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe562dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23339008->23343103] (req 23339008->23343103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760768.824968:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508413824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45844us (46231us total) trans 4294973018 rc 0/0 00000100:00100000:1.0:1789760768.824979:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5730 00000100:00100000:1.0:1789760768.824989:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747826253459 time 747883818679 state 0x2 ref 0 00000001:00080000:2.0:1789760768.825114:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973018 is committed 00000100:00100000:3.0:1789760768.834126:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760768.834258:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508413952 00000100:00100000:0.0:1789760768.834268:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508413952 00000100:00100000:0.0:1789760768.834348:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747826253459 time 747883818679 state 0x0 ref 1 00000100:00100000:0.0:1789760768.834365:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747884079095 time 747883818679 state 0x1 ref 1 00000100:00100000:1.0:1789760768.834434:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747884079095 time 747939141101 state 0x0 ref 1 00000100:00100000:1.0:1789760768.834451:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508413952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760768.834500:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23343104, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:1.0:1789760768.834509:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.834524:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:1.0:1789760768.834528:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.834535:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.834550:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.869507:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23343104 -> 23347200 00010000:00010000:3.0:1789760768.869525:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.869545:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.869555:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.869562:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760768.869567:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.869576:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.869597:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.869605:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe562e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23343104->23347199] (req 23343104->23347199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.869700:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508413952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35249us (35578us total) trans 4294973019 rc 0/0 00000100:00100000:3.0:1789760768.869711:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5731 00000100:00100000:3.0:1789760768.869721:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747884079095 time 747939141101 state 0x2 ref 0 00000001:00080000:0.0:1789760768.871468:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973019 is committed 00000100:00100000:1.0:1789760768.876915:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760768.877248:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508414080 00000100:00100000:2.0:1789760768.877267:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508414080 00000100:00100000:2.0:1789760768.877462:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747884079095 time 747939141101 state 0x0 ref 1 00000100:00100000:2.0:1789760768.877480:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747939401517 time 747939141101 state 0x1 ref 1 00000100:00100000:2.0:1789760768.877512:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747939401517 time 747982222372 state 0x0 ref 1 00000100:00100000:2.0:1789760768.877524:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508414080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760768.877573:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23347200, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:2.0:1789760768.877583:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe562eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.877599:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=747s v=(0 0) 00010000:00010000:2.0:1789760768.877604:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.877611:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe562eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.877626:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe562eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.916549:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973020 is committed 00002000:00010000:0.0:1789760768.916660:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23347200 -> 23351296 00010000:00010000:0.0:1789760768.916676:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe562eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.916689:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe562eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.916698:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe562eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.916711:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760768.916716:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe562eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.916726:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe562eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.916748:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe562eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.916756:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe562eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23347200->23351295] (req 23347200->23351295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.916859:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508414080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39332us (39948us total) trans 4294973020 rc 0/0 00000100:00100000:0.0:1789760768.916879:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5732 00000100:00100000:0.0:1789760768.916888:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747939401517 time 747982222372 state 0x2 ref 0 00000100:00100000:1.0:1789760768.932174:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760768.934102:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508414208 00000100:00100000:0.0:1789760768.934117:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508414208 00000100:00100000:1.0:1789760768.936461:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747939401517 time 747982222372 state 0x0 ref 1 00000100:00100000:1.0:1789760768.936480:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747982482788 time 747982222372 state 0x1 ref 1 00000100:00100000:1.0:1789760768.936512:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747982482788 time 748041222707 state 0x0 ref 1 00000100:00100000:1.0:1789760768.936529:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508414208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760768.936581:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23351296, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:1.0:1789760768.936590:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe562f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.936605:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760768.936610:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.936616:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe562f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.936631:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe562f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.978345:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23351296 -> 23355392 00010000:00010000:3.0:1789760768.978367:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe562f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.978381:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe562f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.978390:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe562f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.978397:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760768.978402:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe562f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.978411:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe562f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.978432:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe562f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.978440:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe562f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23351296->23355391] (req 23351296->23355391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.978528:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508414208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42001us (46361us total) trans 4294973021 rc 0/0 00000100:00100000:3.0:1789760768.978539:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5733 00000100:00100000:3.0:1789760768.978548:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747982482788 time 748041222707 state 0x2 ref 0 00000001:00080000:0.0:1789760768.978725:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973021 is committed 00000100:00100000:3.0:1789760768.988452:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760768.988567:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508414336 00000100:00100000:1.0:1789760768.988577:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508414336 00000100:00100000:1.0:1789760768.988628:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 747982482788 time 748041222707 state 0x0 ref 1 00000100:00100000:1.0:1789760768.988642:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748041483123 time 748041222707 state 0x1 ref 1 00000100:00100000:1.0:1789760768.988671:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748041483123 time 748093382142 state 0x0 ref 1 00000100:00100000:1.0:1789760768.988684:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508414336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760768.988734:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23355392, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:1.0:1789760768.988744:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe562f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.988761:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760768.988765:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760768.988771:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe562f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760768.988786:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe562f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.025514:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23355392 -> 23359488 00010000:00010000:2.0:1789760769.025536:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe562f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.025550:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe562f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.025565:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe562f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.025572:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760769.025577:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe562f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.025586:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe562f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.025604:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe562f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.025611:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe562f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23355392->23359487] (req 23355392->23359487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.025710:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508414336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37022us (37262us total) trans 4294973022 rc 0/0 00000100:00100000:2.0:1789760769.025720:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5734 00000100:00100000:2.0:1789760769.025731:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748041483123 time 748093382142 state 0x2 ref 0 00000001:00080000:1.0:1789760769.025810:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973022 is committed 00000100:00100000:2.0:1789760769.035592:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760769.035786:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508414464 00000100:00100000:0.0:1789760769.035796:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508414464 00000100:00100000:0.0:1789760769.035878:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748041483123 time 748093382142 state 0x0 ref 1 00000100:00100000:0.0:1789760769.035895:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748093642558 time 748093382142 state 0x1 ref 1 00000100:00100000:0.0:1789760769.035972:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748093642558 time 748140682196 state 0x0 ref 1 00000100:00100000:0.0:1789760769.035986:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508414464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760769.036033:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23359488, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:0.0:1789760769.036043:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56300 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.036060:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760769.036064:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.036070:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56300 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.036085:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56300 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760769.036474:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760769.080254:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973023 is committed 00002000:00010000:3.0:1789760769.080740:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23359488 -> 23363584 00010000:00010000:3.0:1789760769.080756:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56300 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.080768:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56300 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.080777:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56300 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.080793:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760769.080954:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56300 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.080964:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56300 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.080984:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56300 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.080991:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56300 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23359488->23363583] (req 23359488->23363583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.081078:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508414464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45089us (45492us total) trans 4294973023 rc 0/0 00000100:00100000:3.0:1789760769.081091:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5735 00000100:00100000:3.0:1789760769.081101:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748093642558 time 748140682196 state 0x2 ref 0 00000100:00100000:0.0:1789760769.085887:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760769.085966:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508414592 00000100:00100000:3.0:1789760769.085976:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508414592 00000100:00100000:3.0:1789760769.086031:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748093642558 time 748140682196 state 0x0 ref 1 00000100:00100000:3.0:1789760769.086046:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748140942612 time 748140682196 state 0x1 ref 1 00000100:00100000:3.0:1789760769.086079:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748140942612 time 748190789921 state 0x0 ref 1 00000100:00100000:3.0:1789760769.086091:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508414592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760769.086136:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23363584, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:3.0:1789760769.086144:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56307 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.086160:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760769.086164:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.086170:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56307 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.086184:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56307 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.108277:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23363584 -> 23367680 00010000:00010000:3.0:1789760769.108319:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56307 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.108333:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56307 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.108342:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56307 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.108350:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760769.108355:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56307 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.108363:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56307 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.108381:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56307 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.108388:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56307 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23363584->23367679] (req 23363584->23367679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.108479:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508414592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22384us (22606us total) trans 4294973024 rc 0/0 00000100:00100000:3.0:1789760769.108489:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5736 00000100:00100000:3.0:1789760769.108498:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748140942612 time 748190789921 state 0x2 ref 0 00000001:00080000:0.0:1789760769.108835:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973024 is committed 00000100:00100000:1.0:1789760769.112484:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760769.112708:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508414720 00000100:00100000:2.0:1789760769.112718:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508414720 00000100:00100000:2.0:1789760769.112776:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748140942612 time 748190789921 state 0x0 ref 1 00000100:00100000:2.0:1789760769.112796:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748191050337 time 748190789921 state 0x1 ref 1 00000100:00100000:2.0:1789760769.112885:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748191050337 time 748217595020 state 0x0 ref 1 00000100:00100000:2.0:1789760769.112899:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508414720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760769.112949:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23367680, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:2.0:1789760769.112958:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5630e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.112975:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760769.112979:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.112986:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5630e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.113001:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5630e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.156715:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973025 is committed 00002000:00010000:0.0:1789760769.156798:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23367680 -> 23371776 00010000:00010000:0.0:1789760769.156813:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5630e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.156824:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5630e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.156832:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5630e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.156843:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760769.156848:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5630e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.156858:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5630e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.156890:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5630e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.156898:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5630e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23367680->23371775] (req 23367680->23371775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.157050:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508414720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44149us (44572us total) trans 4294973025 rc 0/0 00000100:00100000:0.0:1789760769.157060:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5737 00000100:00100000:0.0:1789760769.157070:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748191050337 time 748217595020 state 0x2 ref 0 00000100:00100000:2.0:1789760769.161852:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760769.162030:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508414848 00000100:00100000:3.0:1789760769.162041:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508414848 00000100:00100000:3.0:1789760769.162093:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748191050337 time 748217595020 state 0x0 ref 1 00000100:00100000:3.0:1789760769.162107:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748217855436 time 748217595020 state 0x1 ref 1 00000100:00100000:3.0:1789760769.162136:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748217855436 time 748266846936 state 0x0 ref 1 00000100:00100000:3.0:1789760769.162147:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508414848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760769.162191:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23371776, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:3.0:1789760769.162199:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56315 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.162214:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760769.162218:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.162224:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56315 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.162238:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56315 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.193167:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23371776 -> 23375872 00010000:00010000:3.0:1789760769.193187:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56315 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.193201:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56315 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.193279:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56315 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.193307:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760769.193313:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56315 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.193322:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56315 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.193343:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56315 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.193351:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56315 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23371776->23375871] (req 23371776->23375871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.193455:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508414848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31303us (31607us total) trans 4294973026 rc 0/0 00000100:00100000:3.0:1789760769.193465:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5738 00000100:00100000:3.0:1789760769.193475:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748217855436 time 748266846936 state 0x2 ref 0 00000001:00080000:1.0:1789760769.193949:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973026 is committed 00000100:00100000:1.0:1789760769.202402:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760769.202496:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508414976 00000100:00100000:3.0:1789760769.202505:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508414976 00000100:00100000:3.0:1789760769.202562:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748217855436 time 748266846936 state 0x0 ref 1 00000100:00100000:3.0:1789760769.202577:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748267107352 time 748266846936 state 0x1 ref 1 00000100:00100000:3.0:1789760769.202598:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748267107352 time 748307308986 state 0x0 ref 1 00000100:00100000:3.0:1789760769.202610:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508414976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760769.202657:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23375872, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:3.0:1789760769.202666:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5631c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.202688:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760769.202693:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.202699:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5631c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.202715:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5631c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.237631:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973027 is committed 00002000:00010000:1.0:1789760769.237972:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23375872 -> 23379968 00010000:00010000:1.0:1789760769.237988:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5631c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.238000:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5631c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.238013:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5631c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.238021:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760769.238026:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5631c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.238034:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5631c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.238054:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5631c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.238062:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5631c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23375872->23379967] (req 23375872->23379967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.238192:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508414976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35579us (35795us total) trans 4294973027 rc 0/0 00000100:00100000:1.0:1789760769.238203:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5739 00000100:00100000:1.0:1789760769.238212:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748267107352 time 748307308986 state 0x2 ref 0 00000100:00100000:2.0:1789760769.250919:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760769.251194:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508415104 00000100:00100000:0.0:1789760769.251204:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508415104 00000100:00100000:0.0:1789760769.251474:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748267107352 time 748307308986 state 0x0 ref 1 00000100:00100000:0.0:1789760769.251489:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748307569402 time 748307308986 state 0x1 ref 1 00000100:00100000:0.0:1789760769.251511:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748307569402 time 748356222071 state 0x0 ref 1 00000100:00100000:0.0:1789760769.251529:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508415104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760769.251582:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23379968, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:0.0:1789760769.251591:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56323 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.251607:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760769.251612:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.251619:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56323 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.251633:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56323 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.297754:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23379968 -> 23384064 00010000:00010000:0.0:1789760769.297775:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56323 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.297788:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56323 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.297797:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56323 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.297804:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760769.297809:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56323 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.297818:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56323 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.297840:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56323 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.297848:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56323 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23379968->23384063] (req 23379968->23384063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.298009:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973028 is committed 00000100:00100000:0.0:1789760769.298122:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508415104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46589us (47208us total) trans 4294973028 rc 0/0 00000100:00100000:0.0:1789760769.298133:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5740 00000100:00100000:0.0:1789760769.298145:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748307569402 time 748356222071 state 0x2 ref 0 00000100:00100000:2.0:1789760769.305196:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760769.305456:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508415232 00000100:00100000:3.0:1789760769.305466:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508415232 00000100:00100000:3.0:1789760769.305521:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748307569402 time 748356222071 state 0x0 ref 1 00000100:00100000:3.0:1789760769.305536:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748356482487 time 748356222071 state 0x1 ref 1 00000100:00100000:3.0:1789760769.305624:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748356482487 time 748410334516 state 0x0 ref 1 00000100:00100000:3.0:1789760769.305637:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508415232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760769.305686:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23384064, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:3.0:1789760769.305696:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5632a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.305710:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760769.305714:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.305721:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5632a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.305736:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5632a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.345903:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973029 is committed 00002000:00010000:3.0:1789760769.346058:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23384064 -> 23388160 00010000:00010000:3.0:1789760769.346075:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5632a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.346087:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5632a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.346096:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5632a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.346104:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760769.346108:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5632a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.346453:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5632a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.346478:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5632a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.346486:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5632a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23384064->23388159] (req 23384064->23388159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.346622:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508415232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40982us (41432us total) trans 4294973029 rc 0/0 00000100:00100000:3.0:1789760769.346632:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5741 00000100:00100000:3.0:1789760769.346641:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748356482487 time 748410334516 state 0x2 ref 0 00000100:00100000:1.0:1789760769.350603:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760769.350688:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508415360 00000100:00100000:3.0:1789760769.350697:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508415360 00000100:00100000:3.0:1789760769.350756:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748356482487 time 748410334516 state 0x0 ref 1 00000100:00100000:3.0:1789760769.350779:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748410594932 time 748410334516 state 0x1 ref 1 00000100:00100000:3.0:1789760769.350809:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748410594932 time 748455520027 state 0x0 ref 1 00000100:00100000:3.0:1789760769.350820:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508415360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760769.352082:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23388160, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:3.0:1789760769.352095:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56331 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.352111:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760769.352116:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.352123:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56331 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.352137:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56331 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760769.410843:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23388160 -> 23392256 00010000:00010000:1.0:1789760769.410873:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56331 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.410890:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56331 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.410896:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56331 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.410901:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760769.410904:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56331 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.410910:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56331 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.410927:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56331 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.410935:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56331 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23388160->23392255] (req 23388160->23392255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.411029:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508415360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60206us (60431us total) trans 4294973030 rc 0/0 00000100:00100000:1.0:1789760769.411040:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5742 00000100:00100000:1.0:1789760769.411049:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748410594932 time 748455520027 state 0x2 ref 0 00000001:00080000:3.0:1789760769.411280:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973030 is committed 00000100:00100000:2.0:1789760769.417454:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760769.417497:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508415488 00000100:00100000:0.0:1789760769.417507:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508415488 00000100:00100000:0.0:1789760769.417560:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748410594932 time 748455520027 state 0x0 ref 1 00000100:00100000:0.0:1789760769.417575:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748455780443 time 748455520027 state 0x1 ref 1 00000100:00100000:0.0:1789760769.417604:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748455780443 time 748522315505 state 0x0 ref 1 00000100:00100000:0.0:1789760769.417615:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508415488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760769.417657:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23392256, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:0.0:1789760769.417665:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56338 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.417682:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760769.417686:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.417693:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56338 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.417706:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56338 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.435685:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973031 is committed 00002000:00010000:3.0:1789760769.435822:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23392256 -> 23396352 00010000:00010000:3.0:1789760769.435837:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56338 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.435851:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56338 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.435861:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56338 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.435878:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760769.435883:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56338 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.435892:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56338 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.435911:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56338 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.435924:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56338 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23392256->23396351] (req 23392256->23396351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.436054:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508415488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 18435us (18603us total) trans 4294973031 rc 0/0 00000100:00100000:3.0:1789760769.436065:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5743 00000100:00100000:3.0:1789760769.436075:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748455780443 time 748522315505 state 0x2 ref 0 00000100:00100000:2.0:1789760769.441043:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760769.441721:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508415616 00000100:00100000:3.0:1789760769.441732:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508415616 00000100:00100000:3.0:1789760769.441789:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748455780443 time 748522315505 state 0x0 ref 1 00000100:00100000:3.0:1789760769.441804:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748522575921 time 748522315505 state 0x1 ref 1 00000100:00100000:3.0:1789760769.441837:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748522575921 time 748546547764 state 0x0 ref 1 00000100:00100000:3.0:1789760769.441849:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508415616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760769.442102:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23396352, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:2.0:1789760769.442112:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5633f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.442127:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760769.442131:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.442138:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5633f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.442152:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5633f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.486914:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973032 is committed 00002000:00010000:2.0:1789760769.487006:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23396352 -> 23400448 00010000:00010000:2.0:1789760769.487024:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5633f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.487037:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5633f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.487045:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5633f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.487052:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760769.487056:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5633f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.487065:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5633f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.487090:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5633f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.487098:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5633f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23396352->23400447] (req 23396352->23400447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.487429:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508415616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45578us (46392us total) trans 4294973032 rc 0/0 00000100:00100000:2.0:1789760769.487440:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5744 00000100:00100000:2.0:1789760769.487450:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748522575921 time 748546547764 state 0x2 ref 0 00000100:00100000:1.0:1789760769.497508:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760769.497593:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508415744 00000100:00100000:2.0:1789760769.497603:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508415744 00000100:00100000:2.0:1789760769.497669:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748522575921 time 748546547764 state 0x0 ref 1 00000100:00100000:2.0:1789760769.497683:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748546808180 time 748546547764 state 0x1 ref 1 00000100:00100000:2.0:1789760769.497718:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748546808180 time 748602428871 state 0x0 ref 1 00000100:00100000:2.0:1789760769.497730:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508415744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760769.497777:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23400448, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0xb807 00010000:00010000:2.0:1789760769.497784:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56346 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.497797:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760769.497802:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.497808:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56346 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.497820:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56346 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.564734:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23400448 -> 23404544 00002000:00010000:3.0:1789760769.564743:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760769 -> 1789760770 00002000:00010000:3.0:1789760769.564745:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760769 -> 1789760770 00010000:00010000:3.0:1789760769.564757:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56346 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.564770:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56346 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.564790:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56346 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.564798:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760769.564803:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56346 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.564811:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56346 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.564830:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56346 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.564837:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56346 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23400448->23404543] (req 23400448->23404543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.564937:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508415744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 67204us (67434us total) trans 4294973033 rc 0/0 00000100:00100000:3.0:1789760769.564948:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5745 00000100:00100000:3.0:1789760769.564958:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748546808180 time 748602428871 state 0x2 ref 0 00000001:00080000:3.0:1789760769.564996:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973033 is committed 00000100:00100000:0.0:1789760769.577822:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760769.578681:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508415872 00000100:00100000:3.0:1789760769.578692:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508415872 00000100:00100000:3.0:1789760769.578752:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748546808180 time 748602428871 state 0x0 ref 1 00000100:00100000:3.0:1789760769.578770:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748602689287 time 748602428871 state 0x1 ref 1 00000100:00100000:3.0:1789760769.578812:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748602689287 time 748683522067 state 0x0 ref 1 00000100:00100000:3.0:1789760769.578824:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508415872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760769.578879:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23404544, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:3.0:1789760769.578889:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5634d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.578903:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760769.578907:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.578912:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5634d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.578926:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5634d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760769.635204:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23404544 -> 23408640 00010000:00010000:1.0:1789760769.635232:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5634d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.635245:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5634d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.635254:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5634d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.635262:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760769.635266:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5634d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.635276:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5634d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.635324:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5634d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.635333:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5634d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23404544->23408639] (req 23404544->23408639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.635413:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508415872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56587us (57596us total) trans 4294973034 rc 0/0 00000100:00100000:1.0:1789760769.635423:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5746 00000100:00100000:1.0:1789760769.635433:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748602689287 time 748683522067 state 0x2 ref 0 00000001:00080000:2.0:1789760769.635891:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973034 is committed 00000100:00100000:2.0:1789760769.640206:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760769.640453:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508416000 00000100:00100000:0.0:1789760769.640464:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508416000 00000100:00100000:0.0:1789760769.640519:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748602689287 time 748683522067 state 0x0 ref 1 00000100:00100000:0.0:1789760769.640534:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748683782483 time 748683522067 state 0x1 ref 1 00000100:00100000:0.0:1789760769.640592:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748683782483 time 748745302537 state 0x0 ref 1 00000100:00100000:0.0:1789760769.640605:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508416000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760769.640652:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23408640, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:0.0:1789760769.640661:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56354 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.640676:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:0.0:1789760769.640681:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.640688:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56354 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.640702:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56354 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.670493:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23408640 -> 23412736 00010000:00010000:3.0:1789760769.670516:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56354 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.670529:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56354 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.670538:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56354 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.670551:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760769.670556:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56354 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.670565:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56354 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.670584:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56354 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.670591:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56354 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23408640->23412735] (req 23408640->23412735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.670682:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508416000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30074us (30481us total) trans 4294973035 rc 0/0 00000100:00100000:3.0:1789760769.670693:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5747 00000100:00100000:3.0:1789760769.670703:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748683782483 time 748745302537 state 0x2 ref 0 00000001:00080000:1.0:1789760769.671053:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973035 is committed 00000100:00100000:3.0:1789760769.675968:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760769.676053:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508416128 00000100:00100000:2.0:1789760769.676064:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508416128 00000100:00100000:2.0:1789760769.676120:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748683782483 time 748745302537 state 0x0 ref 1 00000100:00100000:2.0:1789760769.676135:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748745562953 time 748745302537 state 0x1 ref 1 00000100:00100000:2.0:1789760769.676167:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748745562953 time 748780877813 state 0x0 ref 1 00000100:00100000:2.0:1789760769.676181:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508416128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760769.676239:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23412736, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:2.0:1789760769.676248:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5635b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.676263:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760769.676267:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.676274:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5635b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.676322:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5635b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.727632:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973036 is committed 00002000:00010000:2.0:1789760769.727767:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23412736 -> 23416832 00010000:00010000:2.0:1789760769.727857:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5635b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.727881:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5635b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.727891:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5635b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.727899:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760769.727904:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5635b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.727913:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5635b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.727930:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5635b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.727935:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5635b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23412736->23416831] (req 23412736->23416831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.728011:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508416128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51828us (52049us total) trans 4294973036 rc 0/0 00000100:00100000:2.0:1789760769.728037:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5748 00000100:00100000:2.0:1789760769.728046:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748745562953 time 748780877813 state 0x2 ref 0 00000100:00100000:2.0:1789760769.733206:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760769.733738:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508416256 00000100:00100000:1.0:1789760769.733749:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508416256 00000100:00100000:1.0:1789760769.733971:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748745562953 time 748780877813 state 0x0 ref 1 00000100:00100000:1.0:1789760769.733988:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748781138229 time 748780877813 state 0x1 ref 1 00000100:00100000:1.0:1789760769.734071:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748781138229 time 748838779732 state 0x0 ref 1 00000100:00100000:1.0:1789760769.734084:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508416256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760769.734139:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23416832, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:1.0:1789760769.734149:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56362 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.734166:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:1.0:1789760769.734169:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760769.734175:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56362 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.734188:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56362 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.791906:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760769.796673:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973037 is committed 00002000:00010000:3.0:1789760769.796897:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23416832 -> 23420928 00010000:00010000:3.0:1789760769.796914:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56362 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.796927:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56362 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.796938:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56362 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.796945:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760769.796950:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56362 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.796960:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56362 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.796982:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56362 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.796990:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56362 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23416832->23420927] (req 23416832->23420927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.797125:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508416256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 63039us (63924us total) trans 4294973037 rc 0/0 00000100:00100000:3.0:1789760769.797135:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5749 00000100:00100000:3.0:1789760769.797146:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748781138229 time 748838779732 state 0x2 ref 0 00000100:00100000:1.0:1789760769.803078:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760769.803159:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508416384 00000100:00100000:3.0:1789760769.803167:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508416384 00000100:00100000:3.0:1789760769.803276:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748781138229 time 748838779732 state 0x0 ref 1 00000100:00100000:3.0:1789760769.803413:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748839040148 time 748838779732 state 0x1 ref 1 00000100:00100000:3.0:1789760769.803470:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748839040148 time 748908179563 state 0x0 ref 1 00000100:00100000:3.0:1789760769.803482:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508416384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760769.803527:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23420928, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:3.0:1789760769.803536:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56369 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.803551:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:3.0:1789760769.803556:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.803562:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56369 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.803576:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56369 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.857636:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973038 is committed 00002000:00010000:3.0:1789760769.857959:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23420928 -> 23425024 00010000:00010000:3.0:1789760769.857975:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56369 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.857987:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56369 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.857996:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56369 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.858003:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760769.858009:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56369 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.858017:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56369 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.858037:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56369 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.858045:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56369 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23420928->23425023] (req 23420928->23425023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.858132:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508416384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54647us (55058us total) trans 4294973038 rc 0/0 00000100:00100000:3.0:1789760769.858143:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5750 00000100:00100000:3.0:1789760769.858153:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748839040148 time 748908179563 state 0x2 ref 0 00000100:00100000:1.0:1789760769.861719:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760769.861908:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508416512 00000100:00100000:2.0:1789760769.861917:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508416512 00000100:00100000:2.0:1789760769.861965:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748839040148 time 748908179563 state 0x0 ref 1 00000100:00100000:2.0:1789760769.861980:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748908439979 time 748908179563 state 0x1 ref 1 00000100:00100000:2.0:1789760769.862011:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748908439979 time 748966721620 state 0x0 ref 1 00000100:00100000:2.0:1789760769.862023:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508416512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760769.862068:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23425024, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:2.0:1789760769.862077:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56370 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.862093:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=748s v=(0 0) 00010000:00010000:2.0:1789760769.862098:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.862104:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56370 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.862119:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56370 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.894782:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23425024 -> 23429120 00010000:00010000:0.0:1789760769.894801:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56370 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.894813:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56370 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.894822:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56370 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.894829:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760769.894833:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56370 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.894843:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56370 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.894874:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56370 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.894882:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56370 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23425024->23429119] (req 23425024->23429119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.895002:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508416512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32974us (33285us total) trans 4294973039 rc 0/0 00000100:00100000:0.0:1789760769.895016:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5751 00000100:00100000:0.0:1789760769.895037:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748908439979 time 748966721620 state 0x2 ref 0 00000001:00080000:2.0:1789760769.896926:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973039 is committed 00000100:00100000:1.0:1789760769.900736:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760769.901467:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508416640 00000100:00100000:3.0:1789760769.901477:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508416640 00000100:00100000:3.0:1789760769.901529:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748908439979 time 748966721620 state 0x0 ref 1 00000100:00100000:3.0:1789760769.901544:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748966982036 time 748966721620 state 0x1 ref 1 00000100:00100000:3.0:1789760769.901573:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748966982036 time 749006284146 state 0x0 ref 1 00000100:00100000:3.0:1789760769.901583:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508416640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760769.901625:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23429120, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:3.0:1789760769.901634:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56377 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.901649:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760769.901653:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.901667:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56377 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.901682:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56377 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.918906:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534081920:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760769.918978:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760769.919335:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534081920 00000100:00100000:3.0:1789760769.919343:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534081920 00000100:00100000:3.0:1789760769.919389:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:3.0:1789760769.919399:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534081920:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1789760769.919439:0:7358:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:3.0:1789760769.920158:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534081920:12345-0@lo:41:osp-pre-1-0.0 Request processed in 758us (1177us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760769.920167:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00100000:0.0:1789760769.920970:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534081920:0@lo:41:osp-pre-1-0.0 00000001:00080000:0.0:1789760769.929634:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973040 is committed 00002000:00010000:3.0:1789760769.929900:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23429120 -> 23433216 00010000:00010000:3.0:1789760769.929915:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56377 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.929929:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56377 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.929938:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56377 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.929946:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760769.929951:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56377 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.929959:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56377 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.929978:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56377 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.929985:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56377 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23429120->23433215] (req 23429120->23433215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.930078:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508416640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28491us (29346us total) trans 4294973040 rc 0/0 00000100:00100000:3.0:1789760769.930088:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5752 00000100:00100000:3.0:1789760769.930098:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748966982036 time 749006284146 state 0x2 ref 0 00000100:00100000:1.0:1789760769.933350:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760769.934005:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508416768 00000100:00100000:3.0:1789760769.934016:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508416768 00000100:00100000:3.0:1789760769.934077:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 748966982036 time 749006284146 state 0x0 ref 1 00000100:00100000:3.0:1789760769.934093:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749006544562 time 749006284146 state 0x1 ref 1 00000100:00100000:3.0:1789760769.934171:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749006544562 time 749038880590 state 0x0 ref 1 00000100:00100000:3.0:1789760769.934187:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508416768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760769.934245:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23433216, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:3.0:1789760769.934254:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5637e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.934269:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760769.934273:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.934279:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5637e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.934317:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5637e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.957652:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973041 is committed 00002000:00010000:1.0:1789760769.957738:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23433216 -> 23437312 00010000:00010000:1.0:1789760769.957879:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5637e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.957894:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5637e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.957967:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5637e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.957977:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760769.957982:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5637e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.957991:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5637e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.958012:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5637e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760769.958100:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5637e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23433216->23437311] (req 23433216->23437311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760769.958372:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508416768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24182us (25027us total) trans 4294973041 rc 0/0 00000100:00100000:1.0:1789760769.958383:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5753 00000100:00100000:1.0:1789760769.958393:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749006544562 time 749038880590 state 0x2 ref 0 00000100:00100000:0.0:1789760769.963601:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760769.965968:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508416896 00000100:00100000:2.0:1789760769.965982:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508416896 00000100:00100000:2.0:1789760769.966089:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749006544562 time 749038880590 state 0x0 ref 1 00000100:00100000:2.0:1789760769.966108:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749039141006 time 749038880590 state 0x1 ref 1 00000100:00100000:2.0:1789760769.966189:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749039141006 time 749070897899 state 0x0 ref 1 00000100:00100000:2.0:1789760769.966204:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508416896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760769.966263:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23437312, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:2.0:1789760769.966272:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56385 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.966310:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760769.966315:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.966321:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56385 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.966336:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56385 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.991981:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973042 is committed 00002000:00010000:2.0:1789760769.992092:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23437312 -> 23441408 00010000:00010000:2.0:1789760769.992106:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56385 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.992119:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56385 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.992128:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56385 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.992135:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760769.992139:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56385 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.992147:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56385 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.992166:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56385 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.992174:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56385 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23437312->23441407] (req 23437312->23441407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.992281:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508416896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26074us (28684us total) trans 4294973042 rc 0/0 00000100:00100000:2.0:1789760769.992323:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5754 00000100:00100000:2.0:1789760769.992333:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749039141006 time 749070897899 state 0x2 ref 0 00000100:00100000:0.0:1789760769.996181:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760769.996334:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508417024 00000100:00100000:3.0:1789760769.996345:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508417024 00000100:00100000:3.0:1789760769.996398:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749039141006 time 749070897899 state 0x0 ref 1 00000100:00100000:3.0:1789760769.996413:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749071158315 time 749070897899 state 0x1 ref 1 00000100:00100000:3.0:1789760769.996448:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749071158315 time 749101158148 state 0x0 ref 1 00000100:00100000:3.0:1789760769.996460:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508417024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760769.996508:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23441408, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:3.0:1789760769.996516:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5638c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.996534:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760769.996538:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.996544:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5638c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.996559:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5638c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.032307:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973043 is committed 00002000:00010000:3.0:1789760770.032393:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23441408 -> 23445504 00010000:00010000:3.0:1789760770.032407:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5638c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.032420:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5638c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.032429:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5638c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.032436:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760770.032440:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5638c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.032450:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5638c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.032468:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5638c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.032476:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5638c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23441408->23445503] (req 23441408->23445503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.032592:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508417024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36130us (36416us total) trans 4294973043 rc 0/0 00000100:00100000:3.0:1789760770.032602:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5755 00000100:00100000:3.0:1789760770.032611:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749071158315 time 749101158148 state 0x2 ref 0 00000100:00100000:1.0:1789760770.037719:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760770.037916:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508417152 00000100:00100000:3.0:1789760770.037925:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508417152 00000100:00100000:3.0:1789760770.037978:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749071158315 time 749101158148 state 0x0 ref 1 00000100:00100000:3.0:1789760770.037992:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749101418564 time 749101158148 state 0x1 ref 1 00000100:00100000:3.0:1789760770.038061:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749101418564 time 749142770691 state 0x0 ref 1 00000100:00100000:3.0:1789760770.038074:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508417152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760770.038120:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23445504, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:3.0:1789760770.038128:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56393 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.038145:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.038149:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.038154:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56393 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.038173:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56393 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.069269:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23445504 -> 23449600 00010000:00010000:3.0:1789760770.069306:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56393 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.069320:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56393 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.069329:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56393 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.069337:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760770.069342:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56393 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.069352:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56393 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.069371:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56393 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.069379:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56393 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23445504->23449599] (req 23445504->23449599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.069481:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508417152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31404us (31765us total) trans 4294973044 rc 0/0 00000100:00100000:3.0:1789760770.069491:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5756 00000100:00100000:3.0:1789760770.069500:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749101418564 time 749142770691 state 0x2 ref 0 00000001:00080000:0.0:1789760770.069630:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973044 is committed 00000100:00100000:0.0:1789760770.074084:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760770.074175:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508417280 00000100:00100000:2.0:1789760770.074200:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508417280 00000100:00100000:2.0:1789760770.074440:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749101418564 time 749142770691 state 0x0 ref 1 00000100:00100000:2.0:1789760770.074455:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749143031107 time 749142770691 state 0x1 ref 1 00000100:00100000:2.0:1789760770.074522:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749143031107 time 749179228563 state 0x0 ref 1 00000100:00100000:2.0:1789760770.074534:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508417280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760770.074579:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23449600, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:2.0:1789760770.074589:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5639a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.074603:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760770.074607:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.074613:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5639a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.074627:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5639a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.107143:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973045 is committed 00002000:00010000:0.0:1789760770.107365:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23449600 -> 23453696 00010000:00010000:0.0:1789760770.107381:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5639a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.107394:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5639a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.107403:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5639a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.107410:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760770.107414:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5639a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.107423:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5639a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.107444:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5639a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.107450:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe5639a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23449600->23453695] (req 23449600->23453695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.107580:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508417280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33043us (33501us total) trans 4294973045 rc 0/0 00000100:00100000:0.0:1789760770.107590:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5757 00000100:00100000:0.0:1789760770.107599:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749143031107 time 749179228563 state 0x2 ref 0 00000100:00100000:2.0:1789760770.112607:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760770.112713:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508417408 00000100:00100000:3.0:1789760770.112724:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508417408 00000100:00100000:3.0:1789760770.112872:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749143031107 time 749179228563 state 0x0 ref 1 00000100:00100000:3.0:1789760770.112889:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749179488979 time 749179228563 state 0x1 ref 1 00000100:00100000:3.0:1789760770.113031:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749179488979 time 749217739737 state 0x0 ref 1 00000100:00100000:3.0:1789760770.113045:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508417408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760770.113100:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23453696, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:3.0:1789760770.113108:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.113123:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.113128:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.113134:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.113148:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.149386:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973046 is committed 00002000:00010000:3.0:1789760770.149553:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23453696 -> 23457792 00010000:00010000:3.0:1789760770.149570:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.149583:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.149593:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.149600:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760770.149605:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.149615:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.149637:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.149645:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23453696->23457791] (req 23453696->23457791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.149790:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508417408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36741us (37187us total) trans 4294973046 rc 0/0 00000100:00100000:3.0:1789760770.149804:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5758 00000100:00100000:3.0:1789760770.149814:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749179488979 time 749217739737 state 0x2 ref 0 00000100:00100000:1.0:1789760770.153687:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760770.153731:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508417536 00000100:00100000:3.0:1789760770.153741:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508417536 00000100:00100000:3.0:1789760770.153806:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749179488979 time 749217739737 state 0x0 ref 1 00000100:00100000:3.0:1789760770.153821:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749218000153 time 749217739737 state 0x1 ref 1 00000100:00100000:3.0:1789760770.153852:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749218000153 time 749258562211 state 0x0 ref 1 00000100:00100000:3.0:1789760770.153876:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508417536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760770.153919:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23457792, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:3.0:1789760770.153926:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe563a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.153939:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.153942:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.153946:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe563a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.153956:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe563a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.179969:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973047 is committed 00002000:00010000:1.0:1789760770.180106:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23457792 -> 23461888 00010000:00010000:1.0:1789760770.180121:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe563a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.180134:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe563a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.180144:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe563a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.180152:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760770.180157:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe563a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.180165:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe563a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.180183:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe563a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.180190:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe563a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23457792->23461887] (req 23457792->23461887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.180381:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508417536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26503us (26697us total) trans 4294973047 rc 0/0 00000100:00100000:1.0:1789760770.180393:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5759 00000100:00100000:1.0:1789760770.180402:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749218000153 time 749258562211 state 0x2 ref 0 00000100:00100000:0.0:1789760770.185123:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760770.185316:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508417664 00000100:00100000:2.0:1789760770.185325:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508417664 00000100:00100000:2.0:1789760770.185391:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749218000153 time 749258562211 state 0x0 ref 1 00000100:00100000:2.0:1789760770.185407:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749258822627 time 749258562211 state 0x1 ref 1 00000100:00100000:2.0:1789760770.185442:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749258822627 time 749290152210 state 0x0 ref 1 00000100:00100000:2.0:1789760770.185455:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508417664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760770.185501:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23461888, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:2.0:1789760770.185510:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.185527:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760770.185531:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.185537:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.185550:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.250988:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973048 is committed 00002000:00010000:2.0:1789760770.251098:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23461888 -> 23465984 00010000:00010000:2.0:1789760770.251114:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.251138:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.251147:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.251153:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760770.251159:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.251168:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.251189:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.251196:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23461888->23465983] (req 23461888->23465983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.251410:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508417664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 65951us (66288us total) trans 4294973048 rc 0/0 00000100:00100000:2.0:1789760770.251421:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5760 00000100:00100000:2.0:1789760770.251430:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749258822627 time 749290152210 state 0x2 ref 0 00000100:00100000:2.0:1789760770.258273:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760770.258411:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508417792 00000100:00100000:3.0:1789760770.258422:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508417792 00000100:00100000:3.0:1789760770.258476:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749258822627 time 749290152210 state 0x0 ref 1 00000100:00100000:3.0:1789760770.258491:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749290412626 time 749290152210 state 0x1 ref 1 00000100:00100000:3.0:1789760770.258522:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749290412626 time 749363232480 state 0x0 ref 1 00000100:00100000:3.0:1789760770.258535:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508417792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760770.258584:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23465984, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:3.0:1789760770.258594:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.258621:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.258625:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.258632:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.258646:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.313474:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973049 is committed 00002000:00010000:0.0:1789760770.313572:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23465984 -> 23470080 00010000:00010000:0.0:1789760770.313588:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.313601:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.313610:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.313616:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760770.313620:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.313629:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.313648:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.313655:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe563b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23465984->23470079] (req 23465984->23470079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.313745:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508417792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55208us (55477us total) trans 4294973049 rc 0/0 00000100:00100000:0.0:1789760770.313756:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5761 00000100:00100000:0.0:1789760770.313765:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749290412626 time 749363232480 state 0x2 ref 0 00000100:00100000:1.0:1789760770.317074:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760770.317182:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508417920 00000100:00100000:2.0:1789760770.317191:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508417920 00000100:00100000:2.0:1789760770.317256:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749290412626 time 749363232480 state 0x0 ref 1 00000100:00100000:2.0:1789760770.317272:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749363492896 time 749363232480 state 0x1 ref 1 00000100:00100000:2.0:1789760770.317336:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749363492896 time 749422046269 state 0x0 ref 1 00000100:00100000:2.0:1789760770.317349:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508417920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760770.317399:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23470080, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:2.0:1789760770.317409:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.317425:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760770.317429:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.317435:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.317450:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.344632:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973050 is committed 00002000:00010000:2.0:1789760770.344803:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23470080 -> 23474176 00010000:00010000:2.0:1789760770.344818:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.344831:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.344841:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.344849:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760770.344854:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.344872:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.344898:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.344906:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23470080->23474175] (req 23470080->23474175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.345012:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508417920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27659us (27942us total) trans 4294973050 rc 0/0 00000100:00100000:2.0:1789760770.345023:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5762 00000100:00100000:2.0:1789760770.345032:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749363492896 time 749422046269 state 0x2 ref 0 00000100:00100000:1.0:1789760770.349456:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760770.349539:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508418048 00000100:00100000:2.0:1789760770.349550:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508418048 00000100:00100000:2.0:1789760770.349603:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749363492896 time 749422046269 state 0x0 ref 1 00000100:00100000:2.0:1789760770.349619:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749422306685 time 749422046269 state 0x1 ref 1 00000100:00100000:2.0:1789760770.349652:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749422306685 time 749454363131 state 0x0 ref 1 00000100:00100000:2.0:1789760770.349665:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508418048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760770.349711:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23474176, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:2.0:1789760770.349720:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe563c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.349736:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760770.349741:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.349748:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe563c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.349762:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe563c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.379548:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973051 is committed 00002000:00010000:3.0:1789760770.379647:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23474176 -> 23478272 00010000:00010000:3.0:1789760770.379662:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe563c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.379679:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe563c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.379687:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe563c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.379694:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760770.379699:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe563c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.379708:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe563c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.379729:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe563c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.379737:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe563c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23474176->23478271] (req 23474176->23478271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.379905:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508418048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30237us (30453us total) trans 4294973051 rc 0/0 00000100:00100000:3.0:1789760770.379915:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5763 00000100:00100000:3.0:1789760770.379925:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749422306685 time 749454363131 state 0x2 ref 0 00000100:00100000:0.0:1789760770.383102:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760770.383604:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508418176 00000100:00100000:3.0:1789760770.383616:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508418176 00000100:00100000:3.0:1789760770.383671:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749422306685 time 749454363131 state 0x0 ref 1 00000100:00100000:3.0:1789760770.383686:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749454623547 time 749454363131 state 0x1 ref 1 00000100:00100000:3.0:1789760770.383717:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749454623547 time 749488427272 state 0x0 ref 1 00000100:00100000:3.0:1789760770.383729:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508418176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760770.383787:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23478272, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:3.0:1789760770.383796:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.383813:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.383817:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.383824:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.383838:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.412628:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973052 is committed 00002000:00010000:3.0:1789760770.412749:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23478272 -> 23482368 00010000:00010000:3.0:1789760770.412765:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.413079:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.413096:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.413104:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760770.413109:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.413130:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.413152:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.413158:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23478272->23482367] (req 23478272->23482367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.413700:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508418176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29966us (30601us total) trans 4294973052 rc 0/0 00000100:00100000:3.0:1789760770.413712:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5764 00000100:00100000:3.0:1789760770.413723:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749454623547 time 749488427272 state 0x2 ref 0 00000100:00100000:1.0:1789760770.418016:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760770.420811:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508418304 00000100:00100000:2.0:1789760770.420826:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508418304 00000100:00100000:2.0:1789760770.420898:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749454623547 time 749488427272 state 0x0 ref 1 00000100:00100000:2.0:1789760770.420914:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749488687688 time 749488427272 state 0x1 ref 1 00000100:00100000:2.0:1789760770.420937:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749488687688 time 749525647286 state 0x0 ref 1 00000100:00100000:2.0:1789760770.420950:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508418304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760770.420999:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23482368, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:2.0:1789760770.421008:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.421024:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760770.421040:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.421047:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.421062:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760770.454583:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23482368 -> 23486464 00010000:00010000:2.0:1789760770.454604:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.454618:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.454628:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.454635:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760770.454640:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.454649:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.454669:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.454677:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23482368->23486463] (req 23482368->23486463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.454769:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508418304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33818us (36760us total) trans 4294973053 rc 0/0 00000100:00100000:2.0:1789760770.454784:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5765 00000100:00100000:2.0:1789760770.454794:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749488687688 time 749525647286 state 0x2 ref 0 00000001:00080000:0.0:1789760770.455977:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973053 is committed 00000100:00100000:0.0:1789760770.459868:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760770.460021:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508418432 00000100:00100000:3.0:1789760770.460032:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508418432 00000100:00100000:3.0:1789760770.460088:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749488687688 time 749525647286 state 0x0 ref 1 00000100:00100000:3.0:1789760770.460104:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749525907702 time 749525647286 state 0x1 ref 1 00000100:00100000:3.0:1789760770.460186:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749525907702 time 749564895039 state 0x0 ref 1 00000100:00100000:3.0:1789760770.460199:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508418432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760770.460249:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23486464, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:3.0:1789760770.460259:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.460274:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.460279:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.460322:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.460340:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760770.496137:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23486464 -> 23490560 00010000:00010000:1.0:1789760770.496160:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.496173:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.496182:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.496189:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760770.496194:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.496203:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.496237:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.496249:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe563d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23486464->23490559] (req 23486464->23490559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.496382:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508418432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36180us (36531us total) trans 4294973054 rc 0/0 00000100:00100000:1.0:1789760770.496394:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5766 00000100:00100000:1.0:1789760770.496404:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749525907702 time 749564895039 state 0x2 ref 0 00000001:00080000:0.0:1789760770.497008:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973054 is committed 00000100:00100000:1.0:1789760770.503089:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760770.503168:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508418560 00000100:00100000:3.0:1789760770.503177:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508418560 00000100:00100000:3.0:1789760770.503612:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749525907702 time 749564895039 state 0x0 ref 1 00000100:00100000:3.0:1789760770.503629:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749565155455 time 749564895039 state 0x1 ref 1 00000100:00100000:3.0:1789760770.503695:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749565155455 time 749608404729 state 0x0 ref 1 00000100:00100000:3.0:1789760770.503709:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508418560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760770.503760:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23490560, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0xb807 00010000:00010000:3.0:1789760770.503769:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe563e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.503783:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.503788:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.503795:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe563e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.503809:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe563e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760770.536034:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973055 is committed 00002000:00010000:3.0:1789760770.536142:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23490560 -> 23494656 00002000:00010000:3.0:1789760770.536147:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760770 -> 1789760771 00002000:00010000:3.0:1789760770.536150:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760770 -> 1789760771 00010000:00010000:3.0:1789760770.536165:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe563e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.536179:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe563e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.536189:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe563e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.536196:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760770.536201:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe563e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.536319:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe563e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.536340:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe563e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.536348:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe563e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23490560->23494655] (req 23490560->23494655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.536879:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508418560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33166us (33792us total) trans 4294973055 rc 0/0 00000100:00100000:3.0:1789760770.536890:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5767 00000100:00100000:3.0:1789760770.536900:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749565155455 time 749608404729 state 0x2 ref 0 00000100:00100000:3.0:1789760770.544028:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760770.544091:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508418688 00000100:00100000:2.0:1789760770.544101:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508418688 00000100:00100000:2.0:1789760770.544226:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749565155455 time 749608404729 state 0x0 ref 1 00000100:00100000:2.0:1789760770.544243:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749608665145 time 749608404729 state 0x1 ref 1 00000100:00100000:2.0:1789760770.544362:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749608665145 time 749649071387 state 0x0 ref 1 00000100:00100000:2.0:1789760770.544375:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508418688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760770.544423:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23494656, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:2.0:1789760770.544432:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.544448:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760770.544453:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.544459:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.544473:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.585270:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973056 is committed 00002000:00010000:1.0:1789760770.585559:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23494656 -> 23498752 00010000:00010000:1.0:1789760770.585576:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.585590:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.585600:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.585607:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760770.585612:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.585622:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.585644:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.585652:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe563e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23494656->23498751] (req 23494656->23498751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.585848:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508418688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41470us (41825us total) trans 4294973056 rc 0/0 00000100:00100000:1.0:1789760770.585868:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5768 00000100:00100000:1.0:1789760770.585879:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749608665145 time 749649071387 state 0x2 ref 0 00000100:00100000:0.0:1789760770.596626:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760770.596733:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508418816 00000100:00100000:1.0:1789760770.596744:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508418816 00000100:00100000:1.0:1789760770.596981:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749608665145 time 749649071387 state 0x0 ref 1 00000100:00100000:1.0:1789760770.596996:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749649331803 time 749649071387 state 0x1 ref 1 00000100:00100000:1.0:1789760770.597072:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749649331803 time 749701781744 state 0x0 ref 1 00000100:00100000:1.0:1789760770.597085:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508418816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760770.597136:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23498752, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:1.0:1789760770.597145:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.597161:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760770.597166:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.597172:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.597186:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760770.633022:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23498752 -> 23502848 00010000:00010000:1.0:1789760770.633043:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.633055:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.633063:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.633070:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760770.633075:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.633083:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.633103:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.633110:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23498752->23502847] (req 23498752->23502847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.633214:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508418816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36119us (36587us total) trans 4294973057 rc 0/0 00000100:00100000:1.0:1789760770.633225:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5769 00000100:00100000:1.0:1789760770.633234:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749649331803 time 749701781744 state 0x2 ref 0 00000001:00080000:3.0:1789760770.633425:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973057 is committed 00000100:00100000:3.0:1789760770.636530:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760770.636811:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508418944 00000100:00100000:1.0:1789760770.636822:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508418944 00000100:00100000:1.0:1789760770.636932:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749649331803 time 749701781744 state 0x0 ref 1 00000100:00100000:1.0:1789760770.636949:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749702042160 time 749701781744 state 0x1 ref 1 00000100:00100000:1.0:1789760770.636975:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749702042160 time 749741685927 state 0x0 ref 1 00000100:00100000:1.0:1789760770.636987:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508418944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760770.637035:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23502848, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:1.0:1789760770.637044:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe563f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.637085:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760770.637091:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.637098:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe563f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.637112:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe563f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.667618:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973058 is committed 00002000:00010000:1.0:1789760770.667717:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23502848 -> 23506944 00010000:00010000:1.0:1789760770.667733:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe563f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.667746:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe563f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.667755:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe563f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.667762:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760770.667926:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe563f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.667940:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe563f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.667961:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe563f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.667969:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe563f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23502848->23506943] (req 23502848->23506943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.668063:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508418944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31073us (31537us total) trans 4294973058 rc 0/0 00000100:00100000:1.0:1789760770.668077:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5770 00000100:00100000:1.0:1789760770.668088:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749702042160 time 749741685927 state 0x2 ref 0 00000100:00100000:2.0:1789760770.672927:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760770.673005:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508419072 00000100:00100000:1.0:1789760770.673015:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508419072 00000100:00100000:1.0:1789760770.673071:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749702042160 time 749741685927 state 0x0 ref 1 00000100:00100000:1.0:1789760770.673086:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749741946343 time 749741685927 state 0x1 ref 1 00000100:00100000:1.0:1789760770.673118:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749741946343 time 749777827830 state 0x0 ref 1 00000100:00100000:1.0:1789760770.673130:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508419072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760770.673181:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23506944, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:1.0:1789760770.673189:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.673205:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760770.673219:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.673226:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.673240:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.704714:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23506944 -> 23511040 00010000:00010000:3.0:1789760770.704735:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.704748:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.704763:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.704771:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760770.704776:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.704785:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.704805:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.704813:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe563fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23506944->23511039] (req 23506944->23511039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.704918:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508419072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31783us (31995us total) trans 4294973059 rc 0/0 00000100:00100000:3.0:1789760770.704929:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5771 00000100:00100000:3.0:1789760770.704938:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749741946343 time 749777827830 state 0x2 ref 0 00000001:00080000:0.0:1789760770.705633:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973059 is committed 00000100:00100000:2.0:1789760770.711523:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760770.711648:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508419200 00000100:00100000:1.0:1789760770.711659:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508419200 00000100:00100000:1.0:1789760770.711712:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749741946343 time 749777827830 state 0x0 ref 1 00000100:00100000:1.0:1789760770.711728:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749778088246 time 749777827830 state 0x1 ref 1 00000100:00100000:1.0:1789760770.711817:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749778088246 time 749816526002 state 0x0 ref 1 00000100:00100000:1.0:1789760770.711831:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508419200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760770.711898:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23511040, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:1.0:1789760770.711907:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56403 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.711923:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760770.711927:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.711934:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56403 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.711948:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56403 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.751106:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973060 is committed 00002000:00010000:2.0:1789760770.751502:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23511040 -> 23515136 00010000:00010000:2.0:1789760770.751517:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56403 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.751529:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56403 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.751538:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56403 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.751544:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760770.751548:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56403 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.751557:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56403 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.751576:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56403 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.751584:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56403 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23511040->23515135] (req 23511040->23515135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.751676:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508419200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39844us (40160us total) trans 4294973060 rc 0/0 00000100:00100000:2.0:1789760770.751687:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5772 00000100:00100000:2.0:1789760770.751697:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749778088246 time 749816526002 state 0x2 ref 0 00000100:00100000:2.0:1789760770.757621:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760770.757702:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508419328 00000100:00100000:1.0:1789760770.757711:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508419328 00000100:00100000:1.0:1789760770.757773:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749778088246 time 749816526002 state 0x0 ref 1 00000100:00100000:1.0:1789760770.757795:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749816786418 time 749816526002 state 0x1 ref 1 00000100:00100000:1.0:1789760770.757820:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749816786418 time 749862531048 state 0x0 ref 1 00000100:00100000:1.0:1789760770.757832:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508419328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760770.757890:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23515136, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:1.0:1789760770.757898:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5640a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.757914:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:1.0:1789760770.757919:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760770.757925:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5640a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.757938:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5640a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.796821:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973061 is committed 00002000:00010000:2.0:1789760770.796972:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23515136 -> 23519232 00010000:00010000:2.0:1789760770.796987:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5640a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.797000:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5640a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.797009:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5640a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.797015:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760770.797020:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5640a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.797030:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5640a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.797050:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5640a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.797057:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5640a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23515136->23519231] (req 23515136->23519231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.797150:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508419328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39314us (39533us total) trans 4294973061 rc 0/0 00000100:00100000:2.0:1789760770.797172:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5773 00000100:00100000:2.0:1789760770.797182:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749816786418 time 749862531048 state 0x2 ref 0 00000100:00100000:2.0:1789760770.810031:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760770.810679:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508419456 00000100:00100000:3.0:1789760770.810690:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508419456 00000100:00100000:3.0:1789760770.810749:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749816786418 time 749862531048 state 0x0 ref 1 00000100:00100000:3.0:1789760770.810763:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749862791464 time 749862531048 state 0x1 ref 1 00000100:00100000:3.0:1789760770.810800:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749862791464 time 749915509997 state 0x0 ref 1 00000100:00100000:3.0:1789760770.810812:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508419456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760770.810874:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23519232, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:3.0:1789760770.810883:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56411 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.810898:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:3.0:1789760770.810903:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.810909:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56411 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.810923:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56411 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.815323:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760770.861564:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973062 is committed 00002000:00010000:0.0:1789760770.861787:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23519232 -> 23523328 00010000:00010000:0.0:1789760770.861805:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56411 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.861818:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56411 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.861827:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56411 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.861834:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760770.861839:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56411 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.861848:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.861883:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56411 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.861892:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56411 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23519232->23523327] (req 23519232->23523327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.861988:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508419456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51172us (51966us total) trans 4294973062 rc 0/0 00000100:00100000:0.0:1789760770.862003:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5774 00000100:00100000:0.0:1789760770.862013:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749862791464 time 749915509997 state 0x2 ref 0 00000100:00100000:0.0:1789760770.872089:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760770.872929:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508419584 00000100:00100000:2.0:1789760770.872940:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508419584 00000100:00100000:2.0:1789760770.872994:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749862791464 time 749915509997 state 0x0 ref 1 00000100:00100000:2.0:1789760770.873011:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749915770413 time 749915509997 state 0x1 ref 1 00000100:00100000:2.0:1789760770.873047:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749915770413 time 749977757161 state 0x0 ref 1 00000100:00100000:2.0:1789760770.873061:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508419584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760770.873112:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23523328, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:2.0:1789760770.873121:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56418 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.873136:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=749s v=(0 0) 00010000:00010000:2.0:1789760770.873141:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.873148:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56418 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.873163:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56418 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.897254:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973063 is committed 00002000:00010000:1.0:1789760770.897411:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23523328 -> 23527424 00010000:00010000:1.0:1789760770.897425:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56418 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.897438:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56418 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.897448:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56418 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.897455:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760770.897460:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56418 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.897475:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56418 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.897498:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56418 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.897507:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56418 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23523328->23527423] (req 23523328->23527423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.897603:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508419584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24540us (25518us total) trans 4294973063 rc 0/0 00000100:00100000:1.0:1789760770.897613:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5775 00000100:00100000:1.0:1789760770.897623:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749915770413 time 749977757161 state 0x2 ref 0 00000100:00100000:2.0:1789760770.908625:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760770.908750:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508419712 00000100:00100000:0.0:1789760770.908761:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508419712 00000100:00100000:0.0:1789760770.908817:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749915770413 time 749977757161 state 0x0 ref 1 00000100:00100000:0.0:1789760770.908832:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749978017577 time 749977757161 state 0x1 ref 1 00000100:00100000:0.0:1789760770.908966:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749978017577 time 750013674771 state 0x0 ref 1 00000100:00100000:0.0:1789760770.908980:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508419712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760770.909030:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23527424, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:0.0:1789760770.909039:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5641f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.909053:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760770.909058:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.909065:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5641f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.909080:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5641f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.936837:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973064 is committed 00002000:00010000:1.0:1789760770.936990:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23527424 -> 23531520 00010000:00010000:1.0:1789760770.937006:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5641f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.937017:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5641f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.937025:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5641f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.937031:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760770.937037:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5641f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.937046:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5641f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.937068:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5641f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760770.937075:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5641f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23527424->23531519] (req 23527424->23531519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760770.937176:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508419712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28195us (28556us total) trans 4294973064 rc 0/0 00000100:00100000:1.0:1789760770.937186:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5776 00000100:00100000:1.0:1789760770.937196:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749978017577 time 750013674771 state 0x2 ref 0 00000100:00100000:2.0:1789760770.953699:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760770.954481:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508419840 00000100:00100000:0.0:1789760770.954490:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508419840 00000100:00100000:0.0:1789760770.954678:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 749978017577 time 750013674771 state 0x0 ref 1 00000100:00100000:0.0:1789760770.954694:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750013935187 time 750013674771 state 0x1 ref 1 00000100:00100000:0.0:1789760770.954726:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750013935187 time 750059436677 state 0x0 ref 1 00000100:00100000:0.0:1789760770.954738:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508419840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760770.954963:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23531520, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:0.0:1789760770.954972:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56426 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.954988:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760770.954992:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.954999:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56426 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.955014:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56426 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.015454:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23531520 -> 23535616 00010000:00010000:1.0:1789760771.015474:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56426 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.015488:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56426 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.015498:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56426 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.015512:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.015517:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56426 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.015527:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56426 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.015547:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56426 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.015555:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56426 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23531520->23535615] (req 23531520->23535615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.015660:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508419840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60920us (61965us total) trans 4294973065 rc 0/0 00000100:00100000:1.0:1789760771.015670:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5777 00000100:00100000:1.0:1789760771.015680:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750013935187 time 750059436677 state 0x2 ref 0 00000001:00080000:0.0:1789760771.016328:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973065 is committed 00000100:00100000:1.0:1789760771.022938:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760771.023115:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508419968 00000100:00100000:2.0:1789760771.023125:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508419968 00000100:00100000:2.0:1789760771.023228:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750013935187 time 750059436677 state 0x0 ref 1 00000100:00100000:2.0:1789760771.023244:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750059697093 time 750059436677 state 0x1 ref 1 00000100:00100000:2.0:1789760771.023277:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750059697093 time 750127987298 state 0x0 ref 1 00000100:00100000:2.0:1789760771.023308:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508419968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760771.023355:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23535616, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:2.0:1789760771.023364:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5642d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.023380:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760771.023385:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.023391:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5642d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.023404:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5642d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760771.024361:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760771.064107:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973066 is committed 00002000:00010000:0.0:1789760771.064362:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23535616 -> 23539712 00010000:00010000:0.0:1789760771.064378:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5642d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.064391:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5642d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.064401:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5642d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.064408:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760771.064413:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5642d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.064422:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5642d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.064442:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5642d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.064451:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5642d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23535616->23539711] (req 23535616->23539711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.064594:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508419968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41284us (41660us total) trans 4294973066 rc 0/0 00000100:00100000:0.0:1789760771.064604:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5778 00000100:00100000:0.0:1789760771.064614:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750059697093 time 750127987298 state 0x2 ref 0 00000100:00100000:2.0:1789760771.069020:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.069148:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508420096 00000100:00100000:1.0:1789760771.069157:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508420096 00000100:00100000:1.0:1789760771.069211:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750059697093 time 750127987298 state 0x0 ref 1 00000100:00100000:1.0:1789760771.069226:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750128247714 time 750127987298 state 0x1 ref 1 00000100:00100000:1.0:1789760771.069331:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750128247714 time 750174040896 state 0x0 ref 1 00000100:00100000:1.0:1789760771.069344:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508420096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760771.069393:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23539712, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:1.0:1789760771.069401:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56434 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.069416:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.069420:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.069427:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56434 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.069441:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56434 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.144972:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973067 is committed 00002000:00010000:0.0:1789760771.146081:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23539712 -> 23543808 00010000:00010000:0.0:1789760771.146097:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56434 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.146110:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56434 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.146183:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56434 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.146190:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760771.146324:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56434 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.146336:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56434 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.146356:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56434 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.146364:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56434 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23539712->23543807] (req 23539712->23543807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.146653:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760771.147042:0:6460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508420224 00000100:00100000:0.0:1789760771.147395:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508420096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 78048us (78381us total) trans 4294973067 rc 0/0 00000100:00100000:0.0:1789760771.147408:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5779 00000100:00100000:0.0:1789760771.147418:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750128247714 time 750174040896 state 0x2 ref 0 00000100:00100000:1.0:1789760771.147501:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.147532:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.147550:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.147569:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760771.148165:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508420480 00000100:00100000:0.0:1789760771.148175:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508420480 00000100:00100000:2.0:1789760771.148387:0:6460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508420224 00000100:00100000:2.0:1789760771.148449:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 483 00000100:00100000:0.0:1789760771.148500:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508420736 00000100:00100000:0.0:1789760771.148503:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508420736 00000100:00100000:0.0:1789760771.148518:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508420352 00000100:00100000:0.0:1789760771.148520:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508420352 00000100:00100000:0.0:1789760771.148530:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508420608 00000100:00100000:0.0:1789760771.148532:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508420608 00000100:00100000:0.0:1789760771.148543:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 949 00000100:00100000:0.0:1789760771.148549:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508420480:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760771.149872:0:6460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988caf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2408:x1876699508420224:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760771.152939:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508420480:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 4375us (5425us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760771.152963:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 949 00000100:00100000:0.0:1789760771.153002:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 950 00000100:00100000:0.0:1789760771.153018:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508420736:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760771.153468:0:6460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988caf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2408:x1876699508420224:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 5007us (7032us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.153479:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 483 00000100:00100000:2.0:1789760771.153946:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 951 00000100:00100000:2.0:1789760771.153959:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508420352:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760771.154009:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508420352:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 54us (6456us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.154015:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 951 00000100:00100000:2.0:1789760771.154031:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 952 00000100:00100000:2.0:1789760771.154037:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508420608:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760771.154062:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508420608:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 26us (6493us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.154068:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 952 00000100:00100000:3.0:1789760771.156235:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760771.157834:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508420864 00000100:00100000:2.0:1789760771.157848:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508420864 00000100:00100000:2.0:1789760771.157928:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750128247714 time 750174040896 state 0x0 ref 1 00000100:00100000:2.0:1789760771.157943:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750174301312 time 750174040896 state 0x1 ref 1 00000100:00100000:2.0:1789760771.157976:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750174301312 time 750262686365 state 0x0 ref 1 00000100:00100000:2.0:1789760771.157988:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508420864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760771.158036:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23543808, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:2.0:1789760771.158045:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5643b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.158068:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760771.158073:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.158079:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5643b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.158096:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5643b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.158125:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508420736:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 5111us (10587us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760771.158137:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 950 00000100:00080000:0.0:1789760771.158181:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760771.158187:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000001:00080000:1.0:1789760771.189227:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973068 is committed 00002000:00010000:2.0:1789760771.189434:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23543808 -> 23547904 00010000:00010000:2.0:1789760771.189449:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5643b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.189462:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5643b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.189472:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5643b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.189479:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760771.189484:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5643b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.189493:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5643b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.189512:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5643b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.189521:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5643b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23543808->23547903] (req 23543808->23547903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.189613:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508420864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31623us (33383us total) trans 4294973068 rc 0/0 00000100:00100000:2.0:1789760771.189624:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5780 00000100:00100000:2.0:1789760771.189634:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750174301312 time 750262686365 state 0x2 ref 0 00000100:00100000:2.0:1789760771.199963:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760771.200077:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508420992 00000100:00100000:0.0:1789760771.200088:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508420992 00000100:00100000:0.0:1789760771.200145:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750174301312 time 750262686365 state 0x0 ref 1 00000100:00100000:0.0:1789760771.200160:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750262946781 time 750262686365 state 0x1 ref 1 00000100:00100000:0.0:1789760771.200251:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750262946781 time 750304960735 state 0x0 ref 1 00000100:00100000:0.0:1789760771.200264:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508420992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760771.200353:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23547904, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:0.0:1789760771.200362:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56442 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.200379:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760771.200383:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.200390:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56442 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.200405:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56442 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.243898:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23547904 -> 23552000 00010000:00010000:0.0:1789760771.243917:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56442 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.243931:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56442 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.243940:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56442 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.243947:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760771.243952:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56442 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.243962:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56442 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.243981:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56442 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.243988:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56442 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23547904->23551999] (req 23547904->23551999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.244085:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508420992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43818us (44127us total) trans 4294973069 rc 0/0 00000100:00100000:0.0:1789760771.244095:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5781 00000100:00100000:0.0:1789760771.244105:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750262946781 time 750304960735 state 0x2 ref 0 00000001:00080000:1.0:1789760771.244804:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973069 is committed 00000100:00100000:0.0:1789760771.248866:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.248987:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508421120 00000100:00100000:1.0:1789760771.248997:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508421120 00000100:00100000:1.0:1789760771.249054:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750262946781 time 750304960735 state 0x0 ref 1 00000100:00100000:1.0:1789760771.249074:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750305221151 time 750304960735 state 0x1 ref 1 00000100:00100000:1.0:1789760771.249100:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750305221151 time 750353811393 state 0x0 ref 1 00000100:00100000:1.0:1789760771.249111:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508421120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760771.249152:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23552000, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:1.0:1789760771.249161:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56449 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.249175:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.249179:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.249185:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56449 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.249211:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56449 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.302326:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973070 is committed 00002000:00010000:0.0:1789760771.302530:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23552000 -> 23556096 00010000:00010000:0.0:1789760771.302545:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56449 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.302558:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56449 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.302566:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56449 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.302578:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760771.302582:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56449 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.302592:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56449 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.302612:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56449 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.302621:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56449 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23552000->23556095] (req 23552000->23556095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.302717:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508421120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53601us (53870us total) trans 4294973070 rc 0/0 00000100:00100000:0.0:1789760771.302728:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5782 00000100:00100000:0.0:1789760771.302738:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750305221151 time 750353811393 state 0x2 ref 0 00000100:00100000:3.0:1789760771.307858:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760771.308089:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508421248 00000100:00100000:2.0:1789760771.308099:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508421248 00000100:00100000:2.0:1789760771.308149:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750305221151 time 750353811393 state 0x0 ref 1 00000100:00100000:2.0:1789760771.308163:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750354071809 time 750353811393 state 0x1 ref 1 00000100:00100000:2.0:1789760771.308196:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750354071809 time 750412906375 state 0x0 ref 1 00000100:00100000:2.0:1789760771.308208:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508421248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760771.308255:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23556096, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:2.0:1789760771.308263:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56450 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.308279:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760771.308458:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.308467:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56450 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.308482:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56450 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.355712:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973071 is committed 00002000:00010000:1.0:1789760771.355821:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23556096 -> 23560192 00010000:00010000:1.0:1789760771.355837:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56450 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.355870:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56450 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.355880:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56450 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.355894:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.355898:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56450 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.355907:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56450 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.355927:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56450 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.355935:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56450 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23556096->23560191] (req 23556096->23560191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.356024:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508421248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47814us (48180us total) trans 4294973071 rc 0/0 00000100:00100000:1.0:1789760771.356034:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5783 00000100:00100000:1.0:1789760771.356045:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750354071809 time 750412906375 state 0x2 ref 0 00000100:00100000:2.0:1789760771.360534:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760771.360673:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508421376 00000100:00100000:0.0:1789760771.360697:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508421376 00000100:00100000:0.0:1789760771.360759:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750354071809 time 750412906375 state 0x0 ref 1 00000100:00100000:0.0:1789760771.360774:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750413166791 time 750412906375 state 0x1 ref 1 00000100:00100000:0.0:1789760771.360804:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750413166791 time 750465514545 state 0x0 ref 1 00000100:00100000:0.0:1789760771.360819:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508421376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760771.360884:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23560192, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:0.0:1789760771.360893:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56457 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.360909:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760771.360914:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.360920:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56457 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.360934:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56457 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.398193:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973072 is committed 00002000:00010000:1.0:1789760771.398349:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23560192 -> 23564288 00010000:00010000:1.0:1789760771.398365:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56457 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.398377:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56457 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.398391:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56457 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.398397:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.398402:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56457 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.398409:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56457 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.398428:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56457 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.398435:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56457 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23560192->23564287] (req 23560192->23564287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.398562:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508421376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37741us (38033us total) trans 4294973072 rc 0/0 00000100:00100000:1.0:1789760771.398572:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5784 00000100:00100000:1.0:1789760771.398581:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750413166791 time 750465514545 state 0x2 ref 0 00000100:00100000:2.0:1789760771.405148:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760771.405410:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508421504 00000100:00100000:0.0:1789760771.405418:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508421504 00000100:00100000:0.0:1789760771.405471:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750413166791 time 750465514545 state 0x0 ref 1 00000100:00100000:0.0:1789760771.405485:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750465774961 time 750465514545 state 0x1 ref 1 00000100:00100000:0.0:1789760771.405509:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750465774961 time 750510220256 state 0x0 ref 1 00000100:00100000:0.0:1789760771.405521:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d43450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508421504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760771.405570:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23564288, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:0.0:1789760771.405579:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5645e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.405598:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760771.405602:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.405608:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5645e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.405620:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5645e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.451006:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23564288 -> 23568384 00010000:00010000:1.0:1789760771.451026:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5645e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.451045:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5645e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.451054:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5645e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.451061:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.451066:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5645e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.451075:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5645e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.451094:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5645e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.451101:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5645e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23564288->23568383] (req 23564288->23568383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.452559:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d43450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508421504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45681us (46061us total) trans 4294973073 rc 0/0 00000100:00100000:1.0:1789760771.452573:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5785 00000100:00100000:1.0:1789760771.452583:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750465774961 time 750510220256 state 0x2 ref 0 00000001:00080000:2.0:1789760771.453034:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973073 is committed 00000100:00100000:2.0:1789760771.461409:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.462457:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508421632 00000100:00100000:1.0:1789760771.462470:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508421632 00000100:00100000:1.0:1789760771.462524:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750465774961 time 750510220256 state 0x0 ref 1 00000100:00100000:1.0:1789760771.462538:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750510480672 time 750510220256 state 0x1 ref 1 00000100:00100000:1.0:1789760771.462571:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750510480672 time 750567281851 state 0x0 ref 1 00000100:00100000:1.0:1789760771.462585:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508421632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760771.462632:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23568384, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:1.0:1789760771.462641:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56465 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.462657:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.462661:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.462666:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56465 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.462679:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56465 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.489421:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973074 is committed 00002000:00010000:1.0:1789760771.490348:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23568384 -> 23572480 00010000:00010000:1.0:1789760771.490365:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56465 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.490378:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56465 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.490392:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56465 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.490400:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.490405:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56465 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.490482:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56465 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.490502:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56465 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.490509:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56465 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23568384->23572479] (req 23568384->23572479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.490773:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508421632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28185us (29440us total) trans 4294973074 rc 0/0 00000100:00100000:1.0:1789760771.490783:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5786 00000100:00100000:1.0:1789760771.490793:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750510480672 time 750567281851 state 0x2 ref 0 00000100:00100000:3.0:1789760771.495131:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.495237:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508421760 00000100:00100000:1.0:1789760771.495248:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508421760 00000100:00100000:1.0:1789760771.495565:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750510480672 time 750567281851 state 0x0 ref 1 00000100:00100000:1.0:1789760771.495686:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750567542267 time 750567281851 state 0x1 ref 1 00000100:00100000:1.0:1789760771.495782:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750567542267 time 750600491871 state 0x0 ref 1 00000100:00100000:1.0:1789760771.495798:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508421760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760771.495859:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23572480, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0xb807 00010000:00010000:1.0:1789760771.495868:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5646c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.495884:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.495888:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.495894:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5646c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.495910:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5646c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.533675:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23572480 -> 23576576 00002000:00010000:1.0:1789760771.533684:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760771 -> 1789760772 00002000:00010000:1.0:1789760771.533686:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760771 -> 1789760772 00010000:00010000:1.0:1789760771.533699:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5646c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.533713:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5646c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.533722:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5646c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.533730:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.533735:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5646c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.533745:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5646c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.533764:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5646c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.533772:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5646c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23572480->23576575] (req 23572480->23576575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.533945:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508421760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38145us (38816us total) trans 4294973075 rc 0/0 00000100:00100000:1.0:1789760771.533956:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5787 00000100:00100000:1.0:1789760771.533967:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750567542267 time 750600491871 state 0x2 ref 0 00000001:00080000:3.0:1789760771.534677:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973075 is committed 00000100:00100000:2.0:1789760771.538903:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.539019:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508421888 00000100:00100000:1.0:1789760771.539030:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508421888 00000100:00100000:1.0:1789760771.539085:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750567542267 time 750600491871 state 0x0 ref 1 00000100:00100000:1.0:1789760771.539100:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750600752287 time 750600491871 state 0x1 ref 1 00000100:00100000:1.0:1789760771.539131:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750600752287 time 750643841441 state 0x0 ref 1 00000100:00100000:1.0:1789760771.539147:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508421888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760771.539207:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23576576, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:1.0:1789760771.539217:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56473 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.539233:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.539237:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.539243:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56473 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.539256:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56473 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.573178:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23576576 -> 23580672 00010000:00010000:1.0:1789760771.573209:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56473 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.573223:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56473 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.573232:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56473 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.573239:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.573244:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56473 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.573254:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56473 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.573274:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56473 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.573282:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56473 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23576576->23580671] (req 23576576->23580671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.573575:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508421888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34426us (34676us total) trans 4294973076 rc 0/0 00000100:00100000:1.0:1789760771.573586:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5788 00000100:00100000:1.0:1789760771.573596:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750600752287 time 750643841441 state 0x2 ref 0 00000001:00080000:3.0:1789760771.575034:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973076 is committed 00000100:00100000:2.0:1789760771.577811:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.577922:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508422016 00000100:00100000:1.0:1789760771.577933:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508422016 00000100:00100000:1.0:1789760771.577983:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750600752287 time 750643841441 state 0x0 ref 1 00000100:00100000:1.0:1789760771.578000:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750644101857 time 750643841441 state 0x1 ref 1 00000100:00100000:1.0:1789760771.578024:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750644101857 time 750682735254 state 0x0 ref 1 00000100:00100000:1.0:1789760771.578036:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508422016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760771.578087:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23580672, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:1.0:1789760771.578096:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5647a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.578112:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.578116:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.578123:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5647a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.578137:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5647a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.608022:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23580672 -> 23584768 00010000:00010000:1.0:1789760771.608042:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5647a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.608057:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5647a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.608066:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5647a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.608074:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.608078:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5647a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.608088:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5647a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.608106:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5647a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.608115:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5647a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23580672->23584767] (req 23580672->23584767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.608213:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508422016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30174us (30407us total) trans 4294973077 rc 0/0 00000100:00100000:1.0:1789760771.608223:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5789 00000100:00100000:1.0:1789760771.608233:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750644101857 time 750682735254 state 0x2 ref 0 00000001:00080000:3.0:1789760771.610005:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973077 is committed 00000100:00100000:2.0:1789760771.611748:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.617506:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508422144 00000100:00100000:1.0:1789760771.617521:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508422144 00000100:00100000:1.0:1789760771.617581:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750644101857 time 750682735254 state 0x0 ref 1 00000100:00100000:1.0:1789760771.617599:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750682995670 time 750682735254 state 0x1 ref 1 00000100:00100000:1.0:1789760771.617729:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750682995670 time 750722437279 state 0x0 ref 1 00000100:00100000:1.0:1789760771.617743:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508422144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760771.617806:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23584768, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:1.0:1789760771.617814:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56481 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.617830:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.617835:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.617841:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56481 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.617873:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56481 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.654630:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973078 is committed 00002000:00010000:1.0:1789760771.654740:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23584768 -> 23588864 00010000:00010000:1.0:1789760771.654756:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56481 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.654839:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56481 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.654858:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56481 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.654865:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.654869:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56481 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.654877:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56481 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.654895:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56481 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.654903:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56481 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23584768->23588863] (req 23584768->23588863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.655004:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508422144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37260us (43263us total) trans 4294973078 rc 0/0 00000100:00100000:1.0:1789760771.655015:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5790 00000100:00100000:1.0:1789760771.655024:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750682995670 time 750722437279 state 0x2 ref 0 00000100:00100000:0.0:1789760771.658131:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.658207:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508422272 00000100:00100000:1.0:1789760771.658217:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508422272 00000100:00100000:1.0:1789760771.658270:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750682995670 time 750722437279 state 0x0 ref 1 00000100:00100000:1.0:1789760771.658314:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750722697695 time 750722437279 state 0x1 ref 1 00000100:00100000:0.0:1789760771.658421:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750722697695 time 750763128168 state 0x0 ref 1 00000100:00100000:0.0:1789760771.658435:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508422272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760771.658485:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23588864, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:0.0:1789760771.658493:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56488 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.658507:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760771.658512:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.658518:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56488 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.658532:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56488 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.687063:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23588864 -> 23592960 00010000:00010000:0.0:1789760771.687085:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56488 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.687099:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56488 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.687108:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56488 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.687116:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760771.687121:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56488 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.687130:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56488 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.687150:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56488 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.687158:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56488 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23588864->23592959] (req 23588864->23592959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.687270:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508422272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28833us (29144us total) trans 4294973079 rc 0/0 00000100:00100000:0.0:1789760771.687280:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5791 00000100:00100000:0.0:1789760771.687319:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750722697695 time 750763128168 state 0x2 ref 0 00000001:00080000:1.0:1789760771.691147:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973079 is committed 00000100:00100000:3.0:1789760771.691875:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.692001:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508422400 00000100:00100000:1.0:1789760771.692010:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508422400 00000100:00100000:1.0:1789760771.692063:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750722697695 time 750763128168 state 0x0 ref 1 00000100:00100000:1.0:1789760771.692078:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750763388584 time 750763128168 state 0x1 ref 1 00000100:00100000:1.0:1789760771.692111:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750763388584 time 750796822387 state 0x0 ref 1 00000100:00100000:1.0:1789760771.692123:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508422400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760771.692171:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23592960, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:1.0:1789760771.692181:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5648f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.692206:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.692211:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.692218:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5648f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.692232:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5648f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.728119:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973080 is committed 00002000:00010000:3.0:1789760771.728310:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23592960 -> 23597056 00010000:00010000:3.0:1789760771.728327:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5648f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.728405:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5648f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.728415:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5648f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.728422:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760771.728427:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5648f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.728436:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5648f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.728455:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5648f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.728463:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5648f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23592960->23597055] (req 23592960->23597055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.728587:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508422400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36460us (36713us total) trans 4294973080 rc 0/0 00000100:00100000:3.0:1789760771.728597:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5792 00000100:00100000:3.0:1789760771.728608:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750763388584 time 750796822387 state 0x2 ref 0 00000100:00100000:3.0:1789760771.733243:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.734010:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508422528 00000100:00100000:1.0:1789760771.734021:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508422528 00000100:00100000:1.0:1789760771.734079:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750763388584 time 750796822387 state 0x0 ref 1 00000100:00100000:1.0:1789760771.734094:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750797082803 time 750796822387 state 0x1 ref 1 00000100:00100000:1.0:1789760771.734128:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750797082803 time 750838838328 state 0x0 ref 1 00000100:00100000:1.0:1789760771.734142:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508422528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760771.734189:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23597056, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:1.0:1789760771.734264:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56496 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.734281:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.734328:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.734335:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56496 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.734351:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56496 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.782950:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23597056 -> 23601152 00010000:00010000:1.0:1789760771.782970:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56496 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.782983:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56496 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.782992:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56496 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.782999:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.783003:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56496 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.783012:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56496 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.783030:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56496 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.783037:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56496 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23597056->23601151] (req 23597056->23601151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.783113:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508422528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48969us (49874us total) trans 4294973081 rc 0/0 00000100:00100000:1.0:1789760771.783127:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5793 00000100:00100000:1.0:1789760771.783135:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750797082803 time 750838838328 state 0x2 ref 0 00000001:00080000:0.0:1789760771.783213:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973081 is committed 00000100:00100000:0.0:1789760771.788047:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760771.788184:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508422656 00000100:00100000:2.0:1789760771.788199:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508422656 00000100:00100000:2.0:1789760771.788254:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750797082803 time 750838838328 state 0x0 ref 1 00000100:00100000:2.0:1789760771.788269:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750839098744 time 750838838328 state 0x1 ref 1 00000100:00100000:2.0:1789760771.788393:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750839098744 time 750893102834 state 0x0 ref 1 00000100:00100000:2.0:1789760771.788404:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508422656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760771.788448:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23601152, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:2.0:1789760771.788455:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5649d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.788467:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:2.0:1789760771.788471:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.788475:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5649d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.788484:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5649d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.813511:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23601152 -> 23605248 00010000:00010000:2.0:1789760771.813536:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5649d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.813550:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5649d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.813558:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5649d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.813565:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760771.813570:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5649d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.813579:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5649d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.813599:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5649d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.813611:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5649d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23601152->23605247] (req 23601152->23605247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.813703:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508422656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25295us (25661us total) trans 4294973082 rc 0/0 00000001:00080000:0.0:1789760771.813704:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973082 is committed 00000100:00100000:2.0:1789760771.813713:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5794 00000100:00100000:2.0:1789760771.813722:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750839098744 time 750893102834 state 0x2 ref 0 00000100:00100000:3.0:1789760771.823719:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.823817:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508422784 00000100:00100000:1.0:1789760771.823827:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508422784 00000100:00100000:1.0:1789760771.823891:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750839098744 time 750893102834 state 0x0 ref 1 00000100:00100000:1.0:1789760771.823906:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750893363250 time 750893102834 state 0x1 ref 1 00000100:00100000:0.0:1789760771.824049:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750893363250 time 750928756737 state 0x0 ref 1 00000100:00100000:0.0:1789760771.824064:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508422784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760771.824113:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23605248, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:0.0:1789760771.824123:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.824139:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:0.0:1789760771.824143:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.824149:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.824163:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.838719:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760771.859302:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973083 is committed 00002000:00010000:1.0:1789760771.859474:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23605248 -> 23609344 00010000:00010000:1.0:1789760771.859490:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.859502:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.859511:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.859519:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.859523:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.859532:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.859552:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.859562:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23605248->23609343] (req 23605248->23609343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.859656:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508422784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35591us (35943us total) trans 4294973083 rc 0/0 00000100:00100000:1.0:1789760771.859666:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5795 00000100:00100000:1.0:1789760771.859686:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750893363250 time 750928756737 state 0x2 ref 0 00000100:00100000:2.0:1789760771.868863:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.869260:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508422912 00000100:00100000:1.0:1789760771.869270:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508422912 00000100:00100000:1.0:1789760771.869341:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750893363250 time 750928756737 state 0x0 ref 1 00000100:00100000:1.0:1789760771.869357:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750929017153 time 750928756737 state 0x1 ref 1 00000100:00100000:1.0:1789760771.869430:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750929017153 time 750974139481 state 0x0 ref 1 00000100:00100000:1.0:1789760771.869443:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508422912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760771.869489:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23609344, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:1.0:1789760771.869498:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.869513:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=750s v=(0 0) 00010000:00010000:1.0:1789760771.869517:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.869523:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.869542:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.905226:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973084 is committed 00002000:00010000:1.0:1789760771.905694:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23609344 -> 23613440 00010000:00010000:1.0:1789760771.905709:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.905722:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.905731:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.905738:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.905743:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.905752:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.905777:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.905784:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23609344->23613439] (req 23609344->23613439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760771.905887:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508422912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36442us (37036us total) trans 4294973084 rc 0/0 00000100:00100000:1.0:1789760771.905897:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5796 00000100:00100000:1.0:1789760771.905907:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750929017153 time 750974139481 state 0x2 ref 0 00000100:00100000:2.0:1789760771.913621:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760771.913729:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508423040 00000100:00100000:0.0:1789760771.913738:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508423040 00000100:00100000:0.0:1789760771.913806:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750929017153 time 750974139481 state 0x0 ref 1 00000100:00100000:0.0:1789760771.913821:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750974399897 time 750974139481 state 0x1 ref 1 00000100:00100000:0.0:1789760771.913860:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750974399897 time 751018570847 state 0x0 ref 1 00000100:00100000:0.0:1789760771.913873:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508423040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760771.913922:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23613440, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:0.0:1789760771.913930:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe564b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.913945:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760771.913950:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.913956:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe564b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.913970:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe564b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760771.945464:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23613440 -> 23617536 00010000:00010000:1.0:1789760771.945484:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe564b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.945497:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe564b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.945506:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe564b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.945513:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760771.945517:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe564b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.945526:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe564b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.945545:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe564b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.945556:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe564b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23613440->23617535] (req 23613440->23617535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.945758:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973085 is committed 00000100:00100000:1.0:1789760771.945817:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508423040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31941us (32200us total) trans 4294973085 rc 0/0 00000100:00100000:1.0:1789760771.945828:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5797 00000100:00100000:1.0:1789760771.945838:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750974399897 time 751018570847 state 0x2 ref 0 00000100:00100000:3.0:1789760771.958694:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760771.958799:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508423168 00000100:00100000:1.0:1789760771.958809:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508423168 00000100:00100000:1.0:1789760771.958872:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 750974399897 time 751018570847 state 0x0 ref 1 00000100:00100000:1.0:1789760771.958888:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751018831263 time 751018570847 state 0x1 ref 1 00000100:00100000:1.0:1789760771.958916:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751018831263 time 751063626646 state 0x0 ref 1 00000100:00100000:1.0:1789760771.958929:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508423168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760771.958976:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23617536, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:1.0:1789760771.958985:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.959001:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760771.959005:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760771.959011:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760771.959031:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.967149:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3473:1876699534082176:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967222:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967279:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3473:1876699534082688:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967318:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967341:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887e66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3473:1876699534083200:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967362:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967391:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:2.0:1789760771.967397:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3473:1876699534082048:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967423:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967444:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534082560:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967465:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967482:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887e64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534083072:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967502:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967517:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887e65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534083584:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967536:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967560:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534082304:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967583:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967599:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534082816:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967619:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967635:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887e64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534083328:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967656:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967675:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:2.0:1789760771.967681:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3474:1876699534082432:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967700:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967716:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534082944:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967736:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967752:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887e67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534083456:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.967781:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.967822:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534082176 00000100:00100000:2.0:1789760771.967829:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534082176 00000100:00100000:2.0:1789760771.967875:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534082688 00000100:00100000:2.0:1789760771.967878:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534082688 00000100:00100000:2.0:1789760771.967904:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534083200 00000100:00100000:2.0:1789760771.967908:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534083200 00000100:00100000:2.0:1789760771.967925:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534082816 00000100:00100000:2.0:1789760771.967928:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534082816 00000100:00100000:2.0:1789760771.967942:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534083328 00000100:00100000:2.0:1789760771.967944:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534083328 00000100:00100000:2.0:1789760771.967975:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534082432 00000100:00100000:2.0:1789760771.967980:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534082432 00000100:00100000:2.0:1789760771.967999:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 953 00000100:00100000:2.0:1789760771.968010:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3473:x1876699534082176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.968039:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b66df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534083968:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760771.968059:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3473:x1876699534082176:12345-0@lo:400:kworker.0 Request processed in 52us (835us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.968068:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 953 00000100:00100000:3.0:1789760771.968079:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.968086:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 954 00000100:00100000:2.0:1789760771.968094:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3473:x1876699534082688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.968119:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760771.968122:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3473:x1876699534082688:12345-0@lo:400:kworker.0 Request processed in 29us (803us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.968128:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b66c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534083840:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760771.968133:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 954 00000100:00100000:2.0:1789760771.968146:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 955 00000100:00100000:2.0:1789760771.968154:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3473:x1876699534083200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.968164:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.968192:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3473:x1876699534083200:12345-0@lo:400:kworker.0 Request processed in 38us (828us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.968199:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 955 00000100:00100000:3.0:1789760771.968199:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534083968 00000100:00100000:3.0:1789760771.968205:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534083968 00000100:00100000:2.0:1789760771.968216:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 956 00000100:00100000:2.0:1789760771.968226:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534082816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.968241:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534083840 00000100:00100000:2.0:1789760771.968256:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534082816:12345-0@lo:400:kworker.0 Request processed in 33us (636us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.968262:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 956 00000100:00100000:2.0:1789760771.968274:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 957 00000100:00100000:2.0:1789760771.968280:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534083328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.968323:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534083328:12345-0@lo:400:kworker.0 Request processed in 43us (666us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.968329:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 957 00000100:00100000:2.0:1789760771.968340:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 958 00000100:00100000:2.0:1789760771.968346:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3474:x1876699534082432:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.968371:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3474:x1876699534082432:12345-0@lo:400:kworker.0 Request processed in 25us (670us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.968376:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 958 00000100:00100000:2.0:1789760771.968400:0:6459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534082048 00000100:00100000:2.0:1789760771.968405:0:6459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534082048 00000100:00100000:2.0:1789760771.968426:0:6459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 484 00000100:00100000:2.0:1789760771.968431:0:6459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cac850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3473:x1876699534082048:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.968459:0:6459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cac850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3473:x1876699534082048:12345-0@lo:400:kworker.0 Request processed in 27us (1035us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.968464:0:6459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 484 00000100:00100000:2.0:1789760771.968489:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534082560 00000100:00100000:2.0:1789760771.968493:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534082560 00000100:00100000:2.0:1789760771.968513:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534082304 00000100:00100000:2.0:1789760771.968516:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534082304 00000100:00100000:2.0:1789760771.968527:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:2.0:1789760771.968533:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534082560:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.968566:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534082560:12345-0@lo:400:kworker.0 Request processed in 33us (1099us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.968570:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534083840 00000100:00100000:2.0:1789760771.968576:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:2.0:1789760771.968588:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:2.0:1789760771.968595:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534082304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.968622:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534082304:12345-0@lo:400:kworker.0 Request processed in 28us (1039us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.968631:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:2.0:1789760771.968647:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534083072 00000100:00100000:2.0:1789760771.968651:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534083072 00000100:00100000:3.0:1789760771.968681:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:2.0:1789760771.968682:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534083584 00000100:00100000:2.0:1789760771.968687:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534083584 00000100:00100000:3.0:1789760771.968693:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3475:x1876699534083968:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760771.968702:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534082944 00000100:00100000:2.0:1789760771.968707:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534082944 00002000:00000024:3.0:1789760771.968710:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3834642432 avail 3809476608 00000100:00100000:2.0:1789760771.968722:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534083456 00000100:00100000:2.0:1789760771.968727:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534083456 00000100:00100000:2.0:1789760771.968744:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 239 00000100:00100000:2.0:1789760771.968751:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3473:x1876699534083072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.968760:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3475:x1876699534083968:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (680us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.968767:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00100000:3.0:1789760771.968784:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:2.0:1789760771.968789:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534083072:12345-0@lo:400:kworker.0 Request processed in 37us (1285us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.968795:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3475:x1876699534083840:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760771.968804:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 239 00000100:00100000:2.0:1789760771.968820:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 240 00000100:00100000:2.0:1789760771.968827:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3473:x1876699534083584:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.968865:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3473:x1876699534083584:12345-0@lo:400:kworker.0 Request processed in 39us (1328us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.968870:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 240 00000100:00100000:2.0:1789760771.968882:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:2.0:1789760771.968888:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534082944:12345-0@lo:400:kworker.0 00002000:00000024:3.0:1789760771.968906:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760771.968917:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534082944:12345-0@lo:400:kworker.0 Request processed in 29us (1180us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.968924:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00100000:2.0:1789760771.968938:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:2.0:1789760771.968946:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534083456:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.968950:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534083840:12345-0@lo:13:osp-pre-1-0.0 Request processed in 157us (784us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.968962:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:2.0:1789760771.968991:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534083456:12345-0@lo:400:kworker.0 Request processed in 46us (1209us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.968997:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00100000:3.0:1789760771.969024:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b66c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534083840:0@lo:13:osp-pre-1-0.0 00000100:00080000:2.0:1789760771.969048:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760771.969048:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b66df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534083968:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789760771.969055:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789760771.969060:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760771.969062:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760771.969090:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3474:1876699534082432:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.969102:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534082944:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.969112:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887e67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534083456:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.969122:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534082304:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.969132:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534082816:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.969142:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887e64a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534083328:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.969170:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b66dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534083712:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760771.969198:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760771.969231:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3473:1876699534082048:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.969548:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534082560:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.969565:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887e64000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534083072:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.969575:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887e65f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534083584:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.969730:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3473:1876699534082176:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.969741:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3473:1876699534082688:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.969766:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887e66680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3473:1876699534083200:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.973205:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534083712 00000100:00100000:2.0:1789760771.973217:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534083712 00000100:00100000:2.0:1789760771.973264:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:2.0:1789760771.973274:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534083712:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760771.973393:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528285696 pending 1703936 free 3834642432 avail 3809476608 00000100:00100000:2.0:1789760771.973473:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534083712:12345-0@lo:13:osp-pre-0-1.0 Request processed in 199us (4273us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.973482:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:3.0:1789760771.974120:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049be26f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534084096:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760771.974268:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760771.976374:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534084096 00000100:00100000:3.0:1789760771.976386:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534084096 00000100:00100000:3.0:1789760771.976427:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:3.0:1789760771.976435:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534084096:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760771.976456:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760771.976513:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534084096:12345-0@lo:13:osp-pre-1-1.0 Request processed in 79us (2245us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.976520:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:3.0:1789760771.976563:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049be26f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534084096:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760771.976600:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b66dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534083712:0@lo:13:osp-pre-0-1.0 00000001:00080000:0.0:1789760771.998226:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973086 is committed 00002000:00010000:3.0:1789760771.998446:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23617536 -> 23621632 00010000:00010000:3.0:1789760771.998461:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.998476:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.998485:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.998493:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760771.998497:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.998505:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.998531:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.998539:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe564b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23617536->23621631] (req 23617536->23621631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.998642:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508423168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39702us (39945us total) trans 4294973086 rc 0/0 00000100:00100000:3.0:1789760771.998652:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5798 00000100:00100000:3.0:1789760771.998662:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751018831263 time 751063626646 state 0x2 ref 0 00000100:00100000:3.0:1789760772.001718:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760772.003147:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508423296 00000100:00100000:2.0:1789760772.003160:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508423296 00000100:00100000:2.0:1789760772.003541:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751018831263 time 751063626646 state 0x0 ref 1 00000100:00100000:2.0:1789760772.003557:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751063887062 time 751063626646 state 0x1 ref 1 00000100:00100000:2.0:1789760772.003637:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751063887062 time 751108338817 state 0x0 ref 1 00000100:00100000:2.0:1789760772.003651:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508423296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760772.003699:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23621632, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:2.0:1789760772.003709:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe564c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.003729:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760772.003734:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.003740:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe564c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.003754:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe564c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.030813:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973087 is committed 00002000:00010000:3.0:1789760772.030818:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23621632 -> 23625728 00010000:00010000:3.0:1789760772.030833:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe564c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.030857:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe564c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.030866:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe564c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.030874:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760772.030879:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe564c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.030897:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe564c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.030917:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe564c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.030925:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe564c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23621632->23625727] (req 23621632->23625727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.031017:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508423296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27364us (29303us total) trans 4294973087 rc 0/0 00000100:00100000:3.0:1789760772.031027:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5799 00000100:00100000:3.0:1789760772.031036:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751063887062 time 751108338817 state 0x2 ref 0 00000100:00100000:0.0:1789760772.034260:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760772.034415:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508423424 00000100:00100000:1.0:1789760772.034424:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508423424 00000100:00100000:1.0:1789760772.034470:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751063887062 time 751108338817 state 0x0 ref 1 00000100:00100000:1.0:1789760772.034484:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751108599233 time 751108338817 state 0x1 ref 1 00000100:00100000:1.0:1789760772.034543:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751108599233 time 751139253035 state 0x0 ref 1 00000100:00100000:1.0:1789760772.034554:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508423424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760772.034602:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23625728, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:1.0:1789760772.034611:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe564c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.034636:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760772.034640:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.034646:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe564c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.034659:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe564c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.080580:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973088 is committed 00002000:00010000:3.0:1789760772.080698:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23625728 -> 23629824 00010000:00010000:3.0:1789760772.080713:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe564c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.080726:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe564c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.080734:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe564c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.080752:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760772.080757:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe564c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.080766:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe564c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.080785:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe564c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.080792:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe564c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23625728->23629823] (req 23625728->23629823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.080890:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508423424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46332us (46635us total) trans 4294973088 rc 0/0 00000100:00100000:3.0:1789760772.080900:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5800 00000100:00100000:3.0:1789760772.080910:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751108599233 time 751139253035 state 0x2 ref 0 00000100:00100000:0.0:1789760772.085230:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760772.085360:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508423552 00000100:00100000:3.0:1789760772.085369:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508423552 00000100:00100000:3.0:1789760772.085424:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751108599233 time 751139253035 state 0x0 ref 1 00000100:00100000:3.0:1789760772.085440:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751139513451 time 751139253035 state 0x1 ref 1 00000100:00100000:3.0:1789760772.085464:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751139513451 time 751190174359 state 0x0 ref 1 00000100:00100000:3.0:1789760772.085475:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508423552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760772.085523:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23629824, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:3.0:1789760772.085532:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.085549:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.085555:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.085561:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.085575:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760772.119213:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23629824 -> 23633920 00010000:00010000:3.0:1789760772.119234:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.119247:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.119255:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.119262:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760772.119267:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.119276:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.119332:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.119340:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23629824->23633919] (req 23629824->23633919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.119423:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508423552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33945us (34199us total) trans 4294973089 rc 0/0 00000100:00100000:3.0:1789760772.119434:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5801 00000100:00100000:3.0:1789760772.119443:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751139513451 time 751190174359 state 0x2 ref 0 00000001:00080000:0.0:1789760772.120049:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973089 is committed 00000100:00100000:2.0:1789760772.127095:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760772.128158:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508423680 00000100:00100000:3.0:1789760772.128170:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508423680 00000100:00100000:3.0:1789760772.128675:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751139513451 time 751190174359 state 0x0 ref 1 00000100:00100000:3.0:1789760772.128692:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751190434775 time 751190174359 state 0x1 ref 1 00000100:00100000:3.0:1789760772.128781:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751190434775 time 751233490709 state 0x0 ref 1 00000100:00100000:3.0:1789760772.128796:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508423680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760772.128859:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23633920, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:3.0:1789760772.128869:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.128884:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.128889:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.128894:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.128907:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.159172:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973090 is committed 00002000:00010000:3.0:1789760772.159204:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23633920 -> 23638016 00010000:00010000:3.0:1789760772.159220:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.159232:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.159242:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.159250:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760772.159255:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.159264:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.159302:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.159312:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23633920->23638015] (req 23633920->23638015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.159551:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508423680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30753us (32460us total) trans 4294973090 rc 0/0 00000100:00100000:2.0:1789760772.159781:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5802 00000100:00100000:2.0:1789760772.159792:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751190434775 time 751233490709 state 0x2 ref 0 00000100:00100000:2.0:1789760772.165407:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760772.165471:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508423808 00000100:00100000:3.0:1789760772.165481:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508423808 00000100:00100000:3.0:1789760772.165535:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751190434775 time 751233490709 state 0x0 ref 1 00000100:00100000:3.0:1789760772.165551:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751233751125 time 751233490709 state 0x1 ref 1 00000100:00100000:3.0:1789760772.165594:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751233751125 time 751270304080 state 0x0 ref 1 00000100:00100000:3.0:1789760772.165610:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508423808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760772.165657:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23638016, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:3.0:1789760772.165666:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.165682:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.165687:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.165693:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.165706:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.197713:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973091 is committed 00002000:00010000:0.0:1789760772.197797:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23638016 -> 23642112 00010000:00010000:0.0:1789760772.197813:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.197825:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.197838:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.197854:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760772.197859:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.197868:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.197886:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.197894:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe564dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23638016->23642111] (req 23638016->23642111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.198017:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508423808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32408us (32615us total) trans 4294973091 rc 0/0 00000100:00100000:0.0:1789760772.198029:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5803 00000100:00100000:0.0:1789760772.198039:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751233751125 time 751270304080 state 0x2 ref 0 00000100:00100000:0.0:1789760772.200944:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760772.201042:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508423936 00000100:00100000:3.0:1789760772.201051:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508423936 00000100:00100000:3.0:1789760772.201101:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751233751125 time 751270304080 state 0x0 ref 1 00000100:00100000:3.0:1789760772.201119:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751270564496 time 751270304080 state 0x1 ref 1 00000100:00100000:3.0:1789760772.201193:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751270564496 time 751305902635 state 0x0 ref 1 00000100:00100000:3.0:1789760772.201214:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508423936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760772.201260:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23642112, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:3.0:1789760772.201268:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe564e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.201318:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.201323:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.201437:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe564e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.201454:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe564e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.237499:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973092 is committed 00002000:00010000:3.0:1789760772.237665:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23642112 -> 23646208 00010000:00010000:3.0:1789760772.237680:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe564e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.237692:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe564e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.237702:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe564e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.237715:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760772.237720:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe564e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.237729:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe564e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.237748:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe564e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.237945:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe564e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23642112->23646207] (req 23642112->23646207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.238152:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508423936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36935us (37211us total) trans 4294973092 rc 0/0 00000100:00100000:3.0:1789760772.238163:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5804 00000100:00100000:3.0:1789760772.238172:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751270564496 time 751305902635 state 0x2 ref 0 00000100:00100000:2.0:1789760772.249667:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760772.249793:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508424064 00000100:00100000:0.0:1789760772.249803:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508424064 00000100:00100000:0.0:1789760772.250025:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751270564496 time 751305902635 state 0x0 ref 1 00000100:00100000:0.0:1789760772.250037:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751306163051 time 751305902635 state 0x1 ref 1 00000100:00100000:0.0:1789760772.250100:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751306163051 time 751354811085 state 0x0 ref 1 00000100:00100000:0.0:1789760772.250112:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508424064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760772.250157:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23646208, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:0.0:1789760772.250167:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe564ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.250182:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760772.250186:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.250201:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe564ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.250216:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe564ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.287527:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973093 is committed 00002000:00010000:3.0:1789760772.287733:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23646208 -> 23650304 00010000:00010000:3.0:1789760772.287748:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe564ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.287761:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe564ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.287774:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe564ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.287780:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760772.287785:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe564ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.287793:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe564ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.287811:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe564ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.287818:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe564ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23646208->23650303] (req 23646208->23650303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.288026:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508424064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37910us (38362us total) trans 4294973093 rc 0/0 00000100:00100000:3.0:1789760772.288036:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5805 00000100:00100000:3.0:1789760772.288045:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751306163051 time 751354811085 state 0x2 ref 0 00000100:00100000:2.0:1789760772.303221:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760772.304744:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508424192 00000100:00100000:0.0:1789760772.304758:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508424192 00000100:00100000:1.0:1789760772.308302:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751306163051 time 751354811085 state 0x0 ref 1 00000100:00100000:1.0:1789760772.308325:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751355071501 time 751354811085 state 0x1 ref 1 00000100:00100000:1.0:1789760772.308365:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751355071501 time 751413075637 state 0x0 ref 1 00000100:00100000:1.0:1789760772.308389:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508424192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760772.308440:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23650304, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:1.0:1789760772.308449:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe564f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.308465:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:1.0:1789760772.308470:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.308476:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe564f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.308489:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe564f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.358398:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973094 is committed 00002000:00010000:3.0:1789760772.358840:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23650304 -> 23654400 00010000:00010000:3.0:1789760772.358864:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe564f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.358882:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe564f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.358891:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe564f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.358898:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760772.358902:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe564f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.358910:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe564f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.358930:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe564f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.358938:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe564f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23650304->23654399] (req 23650304->23654399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.359826:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508424192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51435us (56608us total) trans 4294973094 rc 0/0 00000100:00100000:3.0:1789760772.359837:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5806 00000100:00100000:3.0:1789760772.359857:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751355071501 time 751413075637 state 0x2 ref 0 00000100:00100000:1.0:1789760772.364802:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760772.364895:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508424320 00000100:00100000:3.0:1789760772.364906:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508424320 00000100:00100000:3.0:1789760772.364969:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751355071501 time 751413075637 state 0x0 ref 1 00000100:00100000:3.0:1789760772.364985:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751413336053 time 751413075637 state 0x1 ref 1 00000100:00100000:3.0:1789760772.365018:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751413336053 time 751469728957 state 0x0 ref 1 00000100:00100000:3.0:1789760772.365031:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508424320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760772.365088:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23654400, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:3.0:1789760772.365098:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe564f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.365113:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.365118:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.365124:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe564f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.365138:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe564f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.396092:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973095 is committed 00002000:00010000:3.0:1789760772.396344:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23654400 -> 23658496 00010000:00010000:3.0:1789760772.396532:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe564f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.396545:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe564f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.396560:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe564f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.396567:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760772.396572:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe564f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.396580:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe564f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.396598:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe564f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.396607:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe564f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23654400->23658495] (req 23654400->23658495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.396697:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508424320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31663us (31899us total) trans 4294973095 rc 0/0 00000100:00100000:3.0:1789760772.396707:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5807 00000100:00100000:3.0:1789760772.396717:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751413336053 time 751469728957 state 0x2 ref 0 00000100:00100000:3.0:1789760772.406620:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760772.407112:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508424448 00000100:00100000:0.0:1789760772.407122:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508424448 00000100:00100000:0.0:1789760772.407175:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751413336053 time 751469728957 state 0x0 ref 1 00000100:00100000:0.0:1789760772.407191:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751469989373 time 751469728957 state 0x1 ref 1 00000100:00100000:0.0:1789760772.407231:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751469989373 time 751511941003 state 0x0 ref 1 00000100:00100000:0.0:1789760772.407244:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508424448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760772.407320:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23658496, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:0.0:1789760772.407329:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.407344:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760772.407348:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.408147:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.408163:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.469513:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973096 is committed 00002000:00010000:3.0:1789760772.473918:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23658496 -> 23662592 00010000:00010000:3.0:1789760772.473939:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.473957:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.473966:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.473973:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760772.473978:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.473987:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.474007:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.474014:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe564ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23658496->23662591] (req 23658496->23662591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.474119:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508424448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 66872us (67504us total) trans 4294973096 rc 0/0 00000100:00100000:3.0:1789760772.474129:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5808 00000100:00100000:3.0:1789760772.474138:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751469989373 time 751511941003 state 0x2 ref 0 00000100:00100000:2.0:1789760772.481367:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760772.481919:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508424576 00000100:00100000:3.0:1789760772.481929:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508424576 00000100:00100000:3.0:1789760772.481998:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751469989373 time 751511941003 state 0x0 ref 1 00000100:00100000:3.0:1789760772.482013:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751512201419 time 751511941003 state 0x1 ref 1 00000100:00100000:3.0:1789760772.482045:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751512201419 time 751586754572 state 0x0 ref 1 00000100:00100000:3.0:1789760772.482058:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508424576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760772.482111:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23662592, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0xb807 00010000:00010000:3.0:1789760772.482121:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56506 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.482137:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.482141:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.482148:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56506 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.482162:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56506 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.511281:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973097 is committed 00002000:00010000:1.0:1789760772.511429:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23662592 -> 23666688 00002000:00010000:1.0:1789760772.511434:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760772 -> 1789760773 00002000:00010000:1.0:1789760772.511436:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760772 -> 1789760773 00010000:00010000:1.0:1789760772.511448:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56506 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.511468:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56506 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.511476:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56506 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.511483:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760772.511488:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56506 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.511497:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56506 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.511515:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56506 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.511522:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56506 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23662592->23666687] (req 23662592->23666687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.511639:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508424576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29578us (30277us total) trans 4294973097 rc 0/0 00000100:00100000:1.0:1789760772.511649:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5809 00000100:00100000:1.0:1789760772.511658:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751512201419 time 751586754572 state 0x2 ref 0 00000100:00100000:1.0:1789760772.520084:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760772.520744:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508424704 00000100:00100000:3.0:1789760772.520755:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508424704 00000100:00100000:2.0:1789760772.521500:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751512201419 time 751586754572 state 0x0 ref 1 00000100:00100000:2.0:1789760772.521518:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751587014988 time 751586754572 state 0x1 ref 1 00000100:00100000:2.0:1789760772.521553:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751587014988 time 751626263243 state 0x0 ref 1 00000100:00100000:2.0:1789760772.521567:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508424704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760772.521630:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23666688, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:2.0:1789760772.521639:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5650d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.521655:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760772.521660:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.521666:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5650d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.521681:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5650d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.565556:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973098 is committed 00002000:00010000:2.0:1789760772.565663:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23666688 -> 23670784 00010000:00010000:2.0:1789760772.565678:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5650d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.565691:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5650d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.565699:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5650d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.565706:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760772.565711:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5650d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.565720:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5650d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.565738:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5650d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.565745:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5650d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23666688->23670783] (req 23666688->23670783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.565877:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508424704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44307us (45797us total) trans 4294973098 rc 0/0 00000100:00100000:2.0:1789760772.565887:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5810 00000100:00100000:2.0:1789760772.565897:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751587014988 time 751626263243 state 0x2 ref 0 00000100:00100000:3.0:1789760772.574274:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760772.574424:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508424832 00000100:00100000:0.0:1789760772.574434:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508424832 00000100:00100000:0.0:1789760772.574485:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751587014988 time 751626263243 state 0x0 ref 1 00000100:00100000:0.0:1789760772.574502:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751626523659 time 751626263243 state 0x1 ref 1 00000100:00100000:0.0:1789760772.574532:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751626523659 time 751679242409 state 0x0 ref 1 00000100:00100000:0.0:1789760772.574543:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e67b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508424832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760772.574591:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23670784, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:0.0:1789760772.574600:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56514 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.574615:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:0.0:1789760772.574619:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.574625:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56514 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.574640:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56514 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.612780:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973099 is committed 00002000:00010000:3.0:1789760772.612892:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23670784 -> 23674880 00010000:00010000:3.0:1789760772.612907:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56514 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.612919:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56514 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.612929:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56514 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.612936:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760772.612941:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56514 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.612950:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56514 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.612970:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56514 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.612978:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56514 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23670784->23674879] (req 23670784->23674879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.613103:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e67b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508424832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38556us (38835us total) trans 4294973099 rc 0/0 00000100:00100000:3.0:1789760772.613113:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5811 00000100:00100000:3.0:1789760772.613121:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751626523659 time 751679242409 state 0x2 ref 0 00000100:00100000:1.0:1789760772.619774:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760772.619834:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508424960 00000100:00100000:3.0:1789760772.619853:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508424960 00000100:00100000:3.0:1789760772.619906:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751626523659 time 751679242409 state 0x0 ref 1 00000100:00100000:3.0:1789760772.619921:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751679502825 time 751679242409 state 0x1 ref 1 00000100:00100000:3.0:1789760772.619955:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751679502825 time 751724664163 state 0x0 ref 1 00000100:00100000:3.0:1789760772.619967:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508424960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760772.620016:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23674880, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:3.0:1789760772.620025:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5651b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.620040:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.620044:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.620049:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5651b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.620063:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5651b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.669424:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973100 is committed 00002000:00010000:3.0:1789760772.669551:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23674880 -> 23678976 00010000:00010000:3.0:1789760772.669577:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5651b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.669591:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5651b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.669600:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5651b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.669608:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760772.669613:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5651b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.669622:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5651b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.669642:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5651b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.669649:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5651b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23674880->23678975] (req 23674880->23678975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.669745:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508424960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49776us (49976us total) trans 4294973100 rc 0/0 00000100:00100000:3.0:1789760772.669755:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5812 00000100:00100000:3.0:1789760772.669766:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751679502825 time 751724664163 state 0x2 ref 0 00000100:00100000:1.0:1789760772.676628:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760772.676718:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508425088 00000100:00100000:2.0:1789760772.676728:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508425088 00000100:00100000:2.0:1789760772.676795:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751679502825 time 751724664163 state 0x0 ref 1 00000100:00100000:2.0:1789760772.676810:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751724924579 time 751724664163 state 0x1 ref 1 00000100:00100000:2.0:1789760772.676838:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751724924579 time 751781548788 state 0x0 ref 1 00000100:00100000:2.0:1789760772.676861:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508425088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760772.676908:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23678976, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:2.0:1789760772.676917:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56522 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.676933:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760772.676938:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.676943:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56522 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.676957:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56522 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760772.728787:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973101 is committed 00002000:00010000:2.0:1789760772.728964:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23678976 -> 23683072 00010000:00010000:2.0:1789760772.728981:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56522 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.728994:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56522 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.729003:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56522 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.729009:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760772.729014:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56522 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.729022:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56522 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.729042:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56522 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.729049:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56522 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23678976->23683071] (req 23678976->23683071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.729197:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508425088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52326us (52566us total) trans 4294973101 rc 0/0 00000100:00100000:2.0:1789760772.729412:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5813 00000100:00100000:2.0:1789760772.729423:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751724924579 time 751781548788 state 0x2 ref 0 00000100:00100000:0.0:1789760772.743188:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760772.743366:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508425216 00000100:00100000:3.0:1789760772.743376:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508425216 00000100:00100000:3.0:1789760772.743442:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751724924579 time 751781548788 state 0x0 ref 1 00000100:00100000:3.0:1789760772.743457:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751781809204 time 751781548788 state 0x1 ref 1 00000100:00100000:3.0:1789760772.743486:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751781809204 time 751848197110 state 0x0 ref 1 00000100:00100000:3.0:1789760772.743498:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508425216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760772.743547:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23683072, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:3.0:1789760772.743557:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56529 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.743582:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.743586:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.743592:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56529 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.743606:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56529 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760772.801531:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23683072 -> 23687168 00010000:00010000:1.0:1789760772.801550:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56529 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.801563:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56529 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.801579:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56529 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.801586:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760772.801591:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56529 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.801600:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56529 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.801620:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56529 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.801627:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56529 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23683072->23687167] (req 23683072->23687167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760772.801724:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508425216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58222us (58540us total) trans 4294973102 rc 0/0 00000100:00100000:1.0:1789760772.801735:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5814 00000100:00100000:1.0:1789760772.801745:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751781809204 time 751848197110 state 0x2 ref 0 00000001:00080000:1.0:1789760772.801780:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973102 is committed 00000100:00100000:0.0:1789760772.806356:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760772.806462:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508425344 00000100:00100000:3.0:1789760772.806472:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508425344 00000100:00100000:3.0:1789760772.806536:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751781809204 time 751848197110 state 0x0 ref 1 00000100:00100000:3.0:1789760772.806552:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751848457526 time 751848197110 state 0x1 ref 1 00000100:00100000:3.0:1789760772.806588:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751848457526 time 751911299268 state 0x0 ref 1 00000100:00100000:3.0:1789760772.806601:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508425344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760772.806650:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23687168, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:3.0:1789760772.806658:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56530 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.806674:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:3.0:1789760772.807169:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.807178:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56530 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.807326:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56530 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.852945:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973103 is committed 00002000:00010000:3.0:1789760772.853136:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23687168 -> 23691264 00010000:00010000:3.0:1789760772.853151:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56530 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.853163:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56530 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.853172:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56530 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.853180:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760772.853185:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56530 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.853267:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56530 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.853315:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56530 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.853323:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56530 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23687168->23691263] (req 23687168->23691263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.853468:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508425344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46863us (47123us total) trans 4294973103 rc 0/0 00000100:00100000:3.0:1789760772.853478:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5815 00000100:00100000:3.0:1789760772.853488:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751848457526 time 751911299268 state 0x2 ref 0 00000100:00100000:3.0:1789760772.858218:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760772.858722:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508425472 00000100:00100000:2.0:1789760772.858732:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508425472 00000100:00100000:2.0:1789760772.858794:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751848457526 time 751911299268 state 0x0 ref 1 00000100:00100000:2.0:1789760772.858809:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751911559684 time 751911299268 state 0x1 ref 1 00000100:00100000:2.0:1789760772.858846:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751911559684 time 751963549190 state 0x0 ref 1 00000100:00100000:2.0:1789760772.858859:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508425472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760772.858902:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23691264, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:2.0:1789760772.858910:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56537 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.858926:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=751s v=(0 0) 00010000:00010000:2.0:1789760772.858930:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.858935:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56537 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.858953:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56537 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.868940:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760772.909343:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23691264 -> 23695360 00010000:00010000:2.0:1789760772.909362:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56537 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.909374:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56537 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.909383:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56537 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.909391:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760772.909395:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56537 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.909403:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56537 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.909423:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56537 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.909431:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56537 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23691264->23695359] (req 23691264->23695359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.909518:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508425472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50657us (51303us total) trans 4294973104 rc 0/0 00000100:00100000:2.0:1789760772.909528:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5816 00000100:00100000:2.0:1789760772.909537:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751911559684 time 751963549190 state 0x2 ref 0 00000001:00080000:2.0:1789760772.909571:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973104 is committed 00000100:00100000:0.0:1789760772.913097:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760772.913369:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508425600 00000100:00100000:1.0:1789760772.913378:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508425600 00000100:00100000:1.0:1789760772.913433:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751911559684 time 751963549190 state 0x0 ref 1 00000100:00100000:1.0:1789760772.913447:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751963809606 time 751963549190 state 0x1 ref 1 00000100:00100000:1.0:1789760772.913511:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751963809606 time 752018220999 state 0x0 ref 1 00000100:00100000:1.0:1789760772.913523:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508425600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760772.913571:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23695360, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:1.0:1789760772.913580:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5653e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.913595:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760772.913600:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760772.913606:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5653e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760772.913623:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5653e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.956162:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973105 is committed 00002000:00010000:2.0:1789760772.956216:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23695360 -> 23699456 00010000:00010000:2.0:1789760772.956235:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5653e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.956248:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5653e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.956256:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5653e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.956263:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760772.956267:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5653e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.956281:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5653e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.956330:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5653e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.956344:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5653e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23695360->23699455] (req 23695360->23699455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.956487:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508425600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42960us (43394us total) trans 4294973105 rc 0/0 00000100:00100000:0.0:1789760772.956498:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5817 00000100:00100000:0.0:1789760772.956506:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751963809606 time 752018220999 state 0x2 ref 0 00000100:00100000:2.0:1789760772.962725:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760772.962848:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508425728 00000100:00100000:3.0:1789760772.962858:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508425728 00000100:00100000:3.0:1789760772.962912:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 751963809606 time 752018220999 state 0x0 ref 1 00000100:00100000:3.0:1789760772.962926:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752018481415 time 752018220999 state 0x1 ref 1 00000100:00100000:3.0:1789760772.962962:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752018481415 time 752067672247 state 0x0 ref 1 00000100:00100000:3.0:1789760772.962974:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508425728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760772.963020:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23699456, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:3.0:1789760772.963029:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56545 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.963043:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760772.963047:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.963053:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56545 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.963067:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56545 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.990485:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760772.990507:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534084224:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760772.990568:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760772.990631:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534084224 00000100:00100000:3.0:1789760772.990638:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534084224 00000100:00100000:3.0:1789760772.990694:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:3.0:1789760772.990703:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534084224:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760772.990744:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789760772.990885:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534084224:12345-0@lo:41:osp-pre-0-1.0 Request processed in 182us (315us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760772.990892:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00100000:0.0:1789760772.991057:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534084224:0@lo:41:osp-pre-0-1.0 00000001:00080000:0.0:1789760772.993512:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973106 is committed 00002000:00010000:2.0:1789760772.993721:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23699456 -> 23703552 00010000:00010000:2.0:1789760772.993735:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56545 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.993747:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56545 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.993824:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56545 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.993832:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760772.993845:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56545 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.993855:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56545 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.993876:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56545 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.993884:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56545 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23699456->23703551] (req 23699456->23703551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.994012:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508425728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31035us (31292us total) trans 4294973106 rc 0/0 00000100:00100000:2.0:1789760772.994022:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5818 00000100:00100000:2.0:1789760772.994031:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752018481415 time 752067672247 state 0x2 ref 0 00000100:00100000:3.0:1789760773.001583:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760773.001804:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508425856 00000100:00100000:2.0:1789760773.001814:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508425856 00000100:00100000:2.0:1789760773.001893:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752018481415 time 752067672247 state 0x0 ref 1 00000100:00100000:2.0:1789760773.001909:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752067932663 time 752067672247 state 0x1 ref 1 00000100:00100000:2.0:1789760773.001943:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752067932663 time 752106653217 state 0x0 ref 1 00000100:00100000:2.0:1789760773.001956:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508425856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760773.002004:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23703552, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:2.0:1789760773.002013:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5654c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.002030:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760773.002034:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.002041:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5654c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.002055:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5654c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760773.002602:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760773.032311:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973107 is committed 00002000:00010000:2.0:1789760773.032559:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23703552 -> 23707648 00010000:00010000:2.0:1789760773.032641:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5654c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.032653:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5654c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.032662:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5654c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.032669:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760773.032674:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5654c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.032682:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5654c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.032702:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5654c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.032710:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5654c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23703552->23707647] (req 23703552->23707647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.032923:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508425856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30966us (31345us total) trans 4294973107 rc 0/0 00000100:00100000:2.0:1789760773.032937:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5819 00000100:00100000:2.0:1789760773.032947:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752067932663 time 752106653217 state 0x2 ref 0 00000100:00100000:0.0:1789760773.044423:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760773.044543:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508425984 00000100:00100000:1.0:1789760773.044554:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508425984 00000100:00100000:1.0:1789760773.044614:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752067932663 time 752106653217 state 0x0 ref 1 00000100:00100000:1.0:1789760773.044630:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752106913633 time 752106653217 state 0x1 ref 1 00000100:00100000:1.0:1789760773.044666:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752106913633 time 752149376511 state 0x0 ref 1 00000100:00100000:1.0:1789760773.044679:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508425984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760773.044736:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23707648, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:1.0:1789760773.044746:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56553 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.044763:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.044768:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.044775:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56553 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.044789:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56553 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.097397:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973108 is committed 00002000:00010000:1.0:1789760773.097529:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23707648 -> 23711744 00010000:00010000:1.0:1789760773.097544:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56553 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.097556:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56553 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.097565:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56553 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.097573:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760773.097577:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56553 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.097586:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56553 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.097605:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56553 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.097613:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56553 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23707648->23711743] (req 23707648->23711743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.097737:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508425984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53055us (53319us total) trans 4294973108 rc 0/0 00000100:00100000:1.0:1789760773.097748:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5820 00000100:00100000:1.0:1789760773.097758:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752106913633 time 752149376511 state 0x2 ref 0 00000100:00100000:0.0:1789760773.102129:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760773.102620:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508426112 00000100:00100000:2.0:1789760773.102632:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508426112 00000100:00100000:2.0:1789760773.102685:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752106913633 time 752149376511 state 0x0 ref 1 00000100:00100000:2.0:1789760773.102699:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752149636927 time 752149376511 state 0x1 ref 1 00000100:00100000:2.0:1789760773.102858:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752149636927 time 752207567273 state 0x0 ref 1 00000100:00100000:2.0:1789760773.102873:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508426112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760773.102923:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23711744, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:2.0:1789760773.102932:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5655a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.102947:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760773.102951:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.102957:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5655a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.102971:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5655a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.146654:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973109 is committed 00002000:00010000:2.0:1789760773.146947:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23711744 -> 23715840 00010000:00010000:2.0:1789760773.146963:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5655a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.146975:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5655a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.146984:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5655a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.146991:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760773.146996:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5655a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.147006:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5655a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.147025:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5655a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.147033:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5655a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23711744->23715839] (req 23711744->23715839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.147114:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508426112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44240us (44991us total) trans 4294973109 rc 0/0 00000100:00100000:2.0:1789760773.147124:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5821 00000100:00100000:2.0:1789760773.147133:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752149636927 time 752207567273 state 0x2 ref 0 00000100:00100000:0.0:1789760773.150660:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760773.150805:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508426240 00000100:00100000:2.0:1789760773.150817:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508426240 00000100:00100000:2.0:1789760773.150881:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752149636927 time 752207567273 state 0x0 ref 1 00000100:00100000:2.0:1789760773.150896:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752207827689 time 752207567273 state 0x1 ref 1 00000100:00100000:2.0:1789760773.150960:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752207827689 time 752255669290 state 0x0 ref 1 00000100:00100000:2.0:1789760773.150973:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508426240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760773.151018:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23715840, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:2.0:1789760773.151027:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56561 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.151042:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760773.151046:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.151051:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56561 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.151064:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56561 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.182135:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973110 is committed 00002000:00010000:2.0:1789760773.182333:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23715840 -> 23719936 00010000:00010000:2.0:1789760773.182350:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56561 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.182363:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56561 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.182373:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56561 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.182380:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760773.182386:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56561 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.182395:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56561 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.182415:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56561 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.182423:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56561 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23715840->23719935] (req 23715840->23719935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.182511:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508426240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31535us (31854us total) trans 4294973110 rc 0/0 00000100:00100000:2.0:1789760773.182521:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5822 00000100:00100000:2.0:1789760773.182530:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752207827689 time 752255669290 state 0x2 ref 0 00000100:00100000:0.0:1789760773.187685:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760773.188034:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508426368 00000100:00100000:1.0:1789760773.188045:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508426368 00000100:00100000:1.0:1789760773.188105:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752207827689 time 752255669290 state 0x0 ref 1 00000100:00100000:1.0:1789760773.188121:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752255929706 time 752255669290 state 0x1 ref 1 00000100:00100000:1.0:1789760773.188201:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752255929706 time 752292910457 state 0x0 ref 1 00000100:00100000:1.0:1789760773.188214:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508426368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760773.188258:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23719936, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:1.0:1789760773.188268:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56568 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.188313:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.188318:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.188325:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56568 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.188352:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56568 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.218562:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973111 is committed 00002000:00010000:1.0:1789760773.218661:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23719936 -> 23724032 00010000:00010000:1.0:1789760773.218676:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56568 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.218690:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56568 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.218700:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56568 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.218708:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760773.218713:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56568 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.218723:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56568 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.218742:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56568 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.218764:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56568 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23719936->23724031] (req 23719936->23724031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.218857:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508426368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30640us (31177us total) trans 4294973111 rc 0/0 00000100:00100000:1.0:1789760773.218868:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5823 00000100:00100000:1.0:1789760773.218877:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752255929706 time 752292910457 state 0x2 ref 0 00000100:00100000:0.0:1789760773.223488:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760773.223575:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508426496 00000100:00100000:3.0:1789760773.223584:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508426496 00000100:00100000:3.0:1789760773.223643:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752255929706 time 752292910457 state 0x0 ref 1 00000100:00100000:3.0:1789760773.223661:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752293170873 time 752292910457 state 0x1 ref 1 00000100:00100000:3.0:1789760773.223691:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752293170873 time 752328400088 state 0x0 ref 1 00000100:00100000:3.0:1789760773.223704:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508426496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760773.223833:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23724032, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:3.0:1789760773.223852:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5656f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.223874:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760773.223878:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.223884:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5656f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.223899:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5656f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.259322:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23724032 -> 23728128 00010000:00010000:2.0:1789760773.259343:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5656f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.259362:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5656f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.259371:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5656f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.259378:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760773.259382:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5656f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.259391:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5656f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.259409:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5656f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.259416:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5656f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23724032->23728127] (req 23724032->23728127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.259503:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508426496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35795us (36019us total) trans 4294973112 rc 0/0 00000100:00100000:2.0:1789760773.259514:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5824 00000100:00100000:2.0:1789760773.259523:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752293170873 time 752328400088 state 0x2 ref 0 00000001:00080000:0.0:1789760773.259619:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973112 is committed 00000100:00100000:0.0:1789760773.267506:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760773.267601:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508426624 00000100:00100000:2.0:1789760773.267612:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508426624 00000100:00100000:2.0:1789760773.267671:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752293170873 time 752328400088 state 0x0 ref 1 00000100:00100000:2.0:1789760773.267686:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752328660504 time 752328400088 state 0x1 ref 1 00000100:00100000:2.0:1789760773.267717:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752328660504 time 752372427059 state 0x0 ref 1 00000100:00100000:2.0:1789760773.267729:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508426624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760773.267779:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23728128, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:2.0:1789760773.267788:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56576 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.267804:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760773.267809:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.267815:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56576 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.267829:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56576 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.314477:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973113 is committed 00002000:00010000:3.0:1789760773.316917:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23728128 -> 23732224 00010000:00010000:3.0:1789760773.316936:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56576 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.316948:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56576 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.316957:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56576 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.316964:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760773.316968:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56576 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.316977:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56576 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.317003:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56576 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.317012:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56576 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23728128->23732223] (req 23728128->23732223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.317169:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508426624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49437us (49668us total) trans 4294973113 rc 0/0 00000100:00100000:3.0:1789760773.317181:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5825 00000100:00100000:3.0:1789760773.317504:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752328660504 time 752372427059 state 0x2 ref 0 00000100:00100000:0.0:1789760773.322741:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760773.323054:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508426752 00000100:00100000:1.0:1789760773.323065:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508426752 00000100:00100000:1.0:1789760773.323123:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752328660504 time 752372427059 state 0x0 ref 1 00000100:00100000:1.0:1789760773.323142:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752372687475 time 752372427059 state 0x1 ref 1 00000100:00100000:1.0:1789760773.323172:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752372687475 time 752427882273 state 0x0 ref 1 00000100:00100000:1.0:1789760773.323193:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508426752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760773.323238:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23732224, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:1.0:1789760773.323247:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5657d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.323262:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.323267:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.323272:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5657d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.323308:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5657d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.363079:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973114 is committed 00002000:00010000:0.0:1789760773.363205:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23732224 -> 23736320 00010000:00010000:0.0:1789760773.363220:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5657d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.363232:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5657d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.363241:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5657d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.363248:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760773.363253:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5657d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.363261:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5657d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.363280:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5657d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.363374:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5657d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23732224->23736319] (req 23732224->23736319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.363464:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508426752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40277us (40729us total) trans 4294973114 rc 0/0 00000100:00100000:0.0:1789760773.363474:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5826 00000100:00100000:0.0:1789760773.363483:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752372687475 time 752427882273 state 0x2 ref 0 00000100:00100000:0.0:1789760773.366753:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760773.366934:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508426880 00000100:00100000:2.0:1789760773.367175:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508426880 00000100:00100000:2.0:1789760773.367230:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752372687475 time 752427882273 state 0x0 ref 1 00000100:00100000:2.0:1789760773.367245:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752428142689 time 752427882273 state 0x1 ref 1 00000100:00100000:2.0:1789760773.367277:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752428142689 time 752471987602 state 0x0 ref 1 00000100:00100000:2.0:1789760773.367333:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508426880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760773.367383:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23736320, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:2.0:1789760773.367391:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56584 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.367407:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:2.0:1789760773.367412:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.367418:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56584 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.367432:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56584 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.418213:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973115 is committed 00002000:00010000:3.0:1789760773.418814:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23736320 -> 23740416 00010000:00010000:3.0:1789760773.418831:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56584 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.418855:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56584 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.418866:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56584 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.418874:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760773.418879:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56584 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.418889:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56584 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.418916:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56584 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.418924:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56584 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23736320->23740415] (req 23736320->23740415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.419021:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508426880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51685us (52272us total) trans 4294973115 rc 0/0 00000100:00100000:3.0:1789760773.419032:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5827 00000100:00100000:3.0:1789760773.419041:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752428142689 time 752471987602 state 0x2 ref 0 00000100:00100000:2.0:1789760773.424354:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760773.424425:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508427008 00000100:00100000:3.0:1789760773.424435:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508427008 00000100:00100000:3.0:1789760773.424487:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752428142689 time 752471987602 state 0x0 ref 1 00000100:00100000:3.0:1789760773.424502:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752472248018 time 752471987602 state 0x1 ref 1 00000100:00100000:3.0:1789760773.424523:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752472248018 time 752529233592 state 0x0 ref 1 00000100:00100000:3.0:1789760773.424535:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508427008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760773.424581:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23740416, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:3.0:1789760773.424590:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5658b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.424605:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760773.424611:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.424618:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5658b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.424633:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5658b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.461508:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973116 is committed 00002000:00010000:0.0:1789760773.461728:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23740416 -> 23744512 00010000:00010000:0.0:1789760773.461742:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5658b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.461755:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5658b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.461764:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5658b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.461771:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760773.461776:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5658b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.461784:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5658b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.461802:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5658b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.461810:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5658b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23740416->23744511] (req 23740416->23744511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.461912:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508427008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37373us (38071us total) trans 4294973116 rc 0/0 00000100:00100000:0.0:1789760773.461922:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5828 00000100:00100000:0.0:1789760773.461931:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752472248018 time 752529233592 state 0x2 ref 0 00000100:00100000:0.0:1789760773.470347:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760773.470451:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508427136 00000100:00100000:1.0:1789760773.470462:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508427136 00000100:00100000:1.0:1789760773.470516:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752472248018 time 752529233592 state 0x0 ref 1 00000100:00100000:1.0:1789760773.470530:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752529494008 time 752529233592 state 0x1 ref 1 00000100:00100000:1.0:1789760773.470562:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752529494008 time 752575272405 state 0x0 ref 1 00000100:00100000:1.0:1789760773.470818:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508427136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760773.470879:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23744512, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0xb807 00010000:00010000:1.0:1789760773.470888:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56592 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.470903:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.470908:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.470914:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56592 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.470927:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56592 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.520767:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973117 is committed 00002000:00010000:1.0:1789760773.520988:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23744512 -> 23748608 00002000:00010000:1.0:1789760773.520993:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760773 -> 1789760774 00002000:00010000:1.0:1789760773.520996:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760773 -> 1789760774 00010000:00010000:1.0:1789760773.521008:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56592 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.521019:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56592 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.521027:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56592 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.521033:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760773.521038:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56592 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.521047:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56592 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.521066:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56592 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.521073:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56592 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23744512->23748607] (req 23744512->23748607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.521256:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508427136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50674us (50912us total) trans 4294973117 rc 0/0 00000100:00100000:1.0:1789760773.521266:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5829 00000100:00100000:1.0:1789760773.521276:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752529494008 time 752575272405 state 0x2 ref 0 00000100:00100000:1.0:1789760773.530703:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760773.530863:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508427264 00000100:00100000:3.0:1789760773.530873:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508427264 00000100:00100000:3.0:1789760773.530928:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752529494008 time 752575272405 state 0x0 ref 1 00000100:00100000:3.0:1789760773.531014:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752575532821 time 752575272405 state 0x1 ref 1 00000100:00100000:3.0:1789760773.531046:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752575532821 time 752635756346 state 0x0 ref 1 00000100:00100000:3.0:1789760773.531059:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508427264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760773.531108:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23748608, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:3.0:1789760773.531117:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56599 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.531133:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:3.0:1789760773.531137:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.531143:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56599 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.531157:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56599 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.560317:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973118 is committed 00002000:00010000:1.0:1789760773.560490:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23748608 -> 23752704 00010000:00010000:1.0:1789760773.560506:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56599 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.560519:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56599 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.560528:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56599 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.560541:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760773.560546:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56599 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.560556:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56599 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.560576:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56599 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.560584:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56599 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23748608->23752703] (req 23748608->23752703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.560677:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508427264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29616us (29979us total) trans 4294973118 rc 0/0 00000100:00100000:1.0:1789760773.560688:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5830 00000100:00100000:1.0:1789760773.560698:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752575532821 time 752635756346 state 0x2 ref 0 00000100:00100000:2.0:1789760773.564435:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760773.564507:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508427392 00000100:00100000:0.0:1789760773.564517:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508427392 00000100:00100000:0.0:1789760773.564568:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752575532821 time 752635756346 state 0x0 ref 1 00000100:00100000:0.0:1789760773.564583:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752636016762 time 752635756346 state 0x1 ref 1 00000100:00100000:0.0:1789760773.564612:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752636016762 time 752669322430 state 0x0 ref 1 00000100:00100000:0.0:1789760773.564625:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508427392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760773.564670:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23752704, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:0.0:1789760773.564679:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe565a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.564698:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760773.564703:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.564709:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe565a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.564721:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe565a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.592041:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973119 is committed 00002000:00010000:1.0:1789760773.592048:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23752704 -> 23756800 00010000:00010000:1.0:1789760773.592062:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe565a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.592075:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe565a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.592091:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe565a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.592098:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760773.592102:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe565a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.592111:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe565a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.592128:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe565a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.592136:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe565a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23752704->23756799] (req 23752704->23756799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.592332:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508427392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27705us (27901us total) trans 4294973119 rc 0/0 00000100:00100000:1.0:1789760773.592343:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5831 00000100:00100000:1.0:1789760773.592352:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752636016762 time 752669322430 state 0x2 ref 0 00000100:00100000:2.0:1789760773.596001:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760773.596109:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508427520 00000100:00100000:1.0:1789760773.596117:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508427520 00000100:00100000:1.0:1789760773.596159:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752636016762 time 752669322430 state 0x0 ref 1 00000100:00100000:1.0:1789760773.596170:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752669582846 time 752669322430 state 0x1 ref 1 00000100:00100000:1.0:1789760773.596202:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752669582846 time 752700912715 state 0x0 ref 1 00000100:00100000:1.0:1789760773.596214:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508427520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760773.596260:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23756800, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:1.0:1789760773.596268:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe565a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.596317:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.596323:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.596330:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe565a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.596344:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe565a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.619088:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973120 is committed 00002000:00010000:1.0:1789760773.619181:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23756800 -> 23760896 00010000:00010000:1.0:1789760773.619267:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe565a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.619282:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe565a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.619310:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe565a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.619318:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760773.619323:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe565a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.619332:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe565a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.619352:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe565a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.619360:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe565a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23756800->23760895] (req 23756800->23760895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.619482:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508427520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23264us (23485us total) trans 4294973120 rc 0/0 00000100:00100000:1.0:1789760773.619492:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5832 00000100:00100000:1.0:1789760773.619501:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752669582846 time 752700912715 state 0x2 ref 0 00000100:00100000:0.0:1789760773.624266:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760773.626379:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508427648 00000100:00100000:1.0:1789760773.626393:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508427648 00000100:00100000:1.0:1789760773.626454:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752669582846 time 752700912715 state 0x0 ref 1 00000100:00100000:1.0:1789760773.626469:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752701173131 time 752700912715 state 0x1 ref 1 00000100:00100000:1.0:1789760773.626501:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752701173131 time 752731211568 state 0x0 ref 1 00000100:00100000:1.0:1789760773.626514:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508427648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760773.626562:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23760896, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:1.0:1789760773.626571:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe565ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.626585:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.626589:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.626595:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe565ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.626609:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe565ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.656130:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973121 is committed 00002000:00010000:0.0:1789760773.656236:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23760896 -> 23764992 00010000:00010000:0.0:1789760773.656251:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe565ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.656263:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe565ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.656273:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe565ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.656280:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760773.656312:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe565ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.656321:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe565ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.656346:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe565ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.656353:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe565ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23760896->23764991] (req 23760896->23764991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.656433:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508427648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29917us (32171us total) trans 4294973121 rc 0/0 00000100:00100000:0.0:1789760773.656443:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5833 00000100:00100000:0.0:1789760773.656453:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752701173131 time 752731211568 state 0x2 ref 0 00000100:00100000:2.0:1789760773.659444:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760773.659526:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508427776 00000100:00100000:0.0:1789760773.659535:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508427776 00000100:00100000:0.0:1789760773.659588:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752701173131 time 752731211568 state 0x0 ref 1 00000100:00100000:0.0:1789760773.659604:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752731471984 time 752731211568 state 0x1 ref 1 00000100:00100000:0.0:1789760773.659632:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752731471984 time 752764343274 state 0x0 ref 1 00000100:00100000:0.0:1789760773.659644:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508427776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760773.659690:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23764992, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:0.0:1789760773.659698:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.659714:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760773.659719:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.659725:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.659738:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.690574:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23764992 -> 23769088 00010000:00010000:0.0:1789760773.690592:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.690605:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.690613:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.690620:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760773.690624:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.690632:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.690649:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.690661:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23764992->23769087] (req 23764992->23769087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.690754:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508427776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31106us (31314us total) trans 4294973122 rc 0/0 00000100:00100000:0.0:1789760773.690764:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5834 00000100:00100000:0.0:1789760773.690775:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752731471984 time 752764343274 state 0x2 ref 0 00000001:00080000:2.0:1789760773.691458:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973122 is committed 00000100:00100000:2.0:1789760773.695117:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760773.696368:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508427904 00000100:00100000:1.0:1789760773.696380:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508427904 00000100:00100000:1.0:1789760773.696434:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752731471984 time 752764343274 state 0x0 ref 1 00000100:00100000:1.0:1789760773.696444:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752764603690 time 752764343274 state 0x1 ref 1 00000100:00100000:1.0:1789760773.696467:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752764603690 time 752801178335 state 0x0 ref 1 00000100:00100000:1.0:1789760773.696480:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508427904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760773.696524:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23769088, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:1.0:1789760773.696532:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.696546:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.696551:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.696582:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.696599:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760773.744888:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23769088 -> 23773184 00010000:00010000:1.0:1789760773.744909:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.744922:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.744931:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.744939:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760773.744944:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.744953:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.744972:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.744981:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23769088->23773183] (req 23769088->23773183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.745080:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508427904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48597us (49967us total) trans 4294973123 rc 0/0 00000100:00100000:1.0:1789760773.745091:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5835 00000100:00100000:1.0:1789760773.745101:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752764603690 time 752801178335 state 0x2 ref 0 00000001:00080000:2.0:1789760773.745195:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973123 is committed 00000100:00100000:1.0:1789760773.748919:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760773.749033:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508428032 00000100:00100000:0.0:1789760773.749043:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508428032 00000100:00100000:0.0:1789760773.749099:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752764603690 time 752801178335 state 0x0 ref 1 00000100:00100000:0.0:1789760773.749113:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752801438751 time 752801178335 state 0x1 ref 1 00000100:00100000:0.0:1789760773.749192:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752801438751 time 752853901694 state 0x0 ref 1 00000100:00100000:0.0:1789760773.749204:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508428032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760773.749250:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23773184, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:0.0:1789760773.749259:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.749274:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760773.749278:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.749320:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.749340:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.785936:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973124 is committed 00002000:00010000:0.0:1789760773.786163:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23773184 -> 23777280 00010000:00010000:0.0:1789760773.786178:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.786191:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.786200:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.786208:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760773.786213:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.786222:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.786334:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.786343:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23773184->23777279] (req 23773184->23777279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.786505:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508428032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37298us (37591us total) trans 4294973124 rc 0/0 00000100:00100000:0.0:1789760773.786517:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5836 00000100:00100000:0.0:1789760773.786527:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752801438751 time 752853901694 state 0x2 ref 0 00000100:00100000:0.0:1789760773.790674:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760773.790775:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508428160 00000100:00100000:1.0:1789760773.790785:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508428160 00000100:00100000:1.0:1789760773.790992:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752801438751 time 752853901694 state 0x0 ref 1 00000100:00100000:1.0:1789760773.791008:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752854162110 time 752853901694 state 0x1 ref 1 00000100:00100000:1.0:1789760773.791082:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752854162110 time 752895791864 state 0x0 ref 1 00000100:00100000:1.0:1789760773.791096:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508428160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760773.791142:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23777280, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:1.0:1789760773.791151:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.791165:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:1.0:1789760773.791169:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.791175:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.791189:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.864306:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973125 is committed 00002000:00010000:1.0:1789760773.864318:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23777280 -> 23781376 00010000:00010000:1.0:1789760773.864334:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.864347:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.864357:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.864365:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760773.864371:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.864381:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.864402:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.864410:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23777280->23781375] (req 23777280->23781375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.864508:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508428160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 73410us (73839us total) trans 4294973125 rc 0/0 00000100:00100000:1.0:1789760773.864521:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5837 00000100:00100000:1.0:1789760773.864531:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752854162110 time 752895791864 state 0x2 ref 0 00000100:00100000:2.0:1789760773.868547:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760773.870709:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508428288 00000100:00100000:1.0:1789760773.870722:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508428288 00000100:00100000:0.0:1789760773.873380:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752854162110 time 752895791864 state 0x0 ref 1 00000100:00100000:0.0:1789760773.873398:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752896052280 time 752895791864 state 0x1 ref 1 00000100:00100000:0.0:1789760773.873505:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752896052280 time 752978214845 state 0x0 ref 1 00000100:00100000:0.0:1789760773.873520:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508428288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760773.873569:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23781376, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:0.0:1789760773.873578:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.873594:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=752s v=(0 0) 00010000:00010000:0.0:1789760773.873599:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.873605:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.873619:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.886459:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760773.913757:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23781376 -> 23785472 00010000:00010000:0.0:1789760773.913777:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.913791:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.913799:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.913806:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760773.913811:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.913820:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.913855:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.913863:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23781376->23785471] (req 23781376->23785471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.913954:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508428288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40433us (45411us total) trans 4294973126 rc 0/0 00000100:00100000:0.0:1789760773.913969:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5838 00000100:00100000:0.0:1789760773.913978:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752896052280 time 752978214845 state 0x2 ref 0 00000001:00080000:1.0:1789760773.914409:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973126 is committed 00000100:00100000:0.0:1789760773.918659:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760773.918775:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508428416 00000100:00100000:2.0:1789760773.918785:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508428416 00000100:00100000:2.0:1789760773.918846:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752896052280 time 752978214845 state 0x0 ref 1 00000100:00100000:2.0:1789760773.918862:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752978475261 time 752978214845 state 0x1 ref 1 00000100:00100000:2.0:1789760773.918934:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752978475261 time 753023643272 state 0x0 ref 1 00000100:00100000:2.0:1789760773.918946:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508428416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760773.918995:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23785472, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:2.0:1789760773.919004:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe565d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.919019:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760773.919023:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.919029:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe565d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.919043:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe565d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.946905:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973127 is committed 00002000:00010000:0.0:1789760773.947754:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23785472 -> 23789568 00010000:00010000:0.0:1789760773.947772:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe565d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.947785:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe565d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.947794:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe565d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.947802:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760773.947807:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe565d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.947816:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe565d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.947846:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe565d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.947855:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe565d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23785472->23789567] (req 23785472->23789567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.947964:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508428416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29015us (29310us total) trans 4294973127 rc 0/0 00000100:00100000:0.0:1789760773.947974:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5839 00000100:00100000:0.0:1789760773.947984:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752978475261 time 753023643272 state 0x2 ref 0 00000100:00100000:2.0:1789760773.958562:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760773.959057:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508428544 00000100:00100000:1.0:1789760773.959069:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508428544 00000100:00100000:1.0:1789760773.959129:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 752978475261 time 753023643272 state 0x0 ref 1 00000100:00100000:1.0:1789760773.959145:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753023903688 time 753023643272 state 0x1 ref 1 00000100:00100000:1.0:1789760773.959179:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753023903688 time 753063889342 state 0x0 ref 1 00000100:00100000:1.0:1789760773.959192:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508428544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760773.959241:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23789568, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:1.0:1789760773.959250:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.959267:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760773.959272:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.959278:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.959327:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.007673:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23789568 -> 23793664 00010000:00010000:0.0:1789760774.007692:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.007705:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.007714:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.007720:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760774.007724:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.007732:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.007751:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.007763:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe565df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23789568->23793663] (req 23789568->23793663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.007864:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508428544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48670us (49306us total) trans 4294973128 rc 0/0 00000100:00100000:0.0:1789760774.007874:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5840 00000100:00100000:0.0:1789760774.007884:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753023903688 time 753063889342 state 0x2 ref 0 00000001:00080000:3.0:1789760774.007960:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973128 is committed 00000100:00100000:2.0:1789760774.011972:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760774.012056:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508428672 00000100:00100000:0.0:1789760774.012065:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508428672 00000100:00100000:0.0:1789760774.012115:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753023903688 time 753063889342 state 0x0 ref 1 00000100:00100000:0.0:1789760774.012129:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753064149758 time 753063889342 state 0x1 ref 1 00000100:00100000:0.0:1789760774.012159:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753064149758 time 753116870401 state 0x0 ref 1 00000100:00100000:0.0:1789760774.012171:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508428672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760774.012219:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23793664, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:0.0:1789760774.012227:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.012244:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760774.012248:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.012255:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.012273:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.055443:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973129 is committed 00002000:00010000:2.0:1789760774.055573:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23793664 -> 23797760 00010000:00010000:2.0:1789760774.055589:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.055602:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.055617:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.055624:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760774.055628:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.055636:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.055654:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.055662:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe565e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23793664->23797759] (req 23793664->23797759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.055752:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508428672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43578us (43786us total) trans 4294973129 rc 0/0 00000100:00100000:2.0:1789760774.055762:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5841 00000100:00100000:2.0:1789760774.055771:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753064149758 time 753116870401 state 0x2 ref 0 00000100:00100000:2.0:1789760774.060571:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760774.060655:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508428800 00000100:00100000:0.0:1789760774.060665:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508428800 00000100:00100000:0.0:1789760774.060715:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753064149758 time 753116870401 state 0x0 ref 1 00000100:00100000:0.0:1789760774.060729:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753117130817 time 753116870401 state 0x1 ref 1 00000100:00100000:0.0:1789760774.060758:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753117130817 time 753165469258 state 0x0 ref 1 00000100:00100000:0.0:1789760774.060769:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d42450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508428800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760774.060816:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23797760, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:0.0:1789760774.060825:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.060850:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760774.060854:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.060860:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.060874:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.097562:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23797760 -> 23801856 00010000:00010000:0.0:1789760774.097583:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.097595:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.097602:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.097609:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760774.097613:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.097621:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.097640:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.097646:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe565ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23797760->23801855] (req 23797760->23801855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.097737:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d42450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508428800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36964us (37171us total) trans 4294973130 rc 0/0 00000100:00100000:0.0:1789760774.097753:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5842 00000100:00100000:0.0:1789760774.097763:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753117130817 time 753165469258 state 0x2 ref 0 00000001:00080000:3.0:1789760774.100162:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973130 is committed 00000100:00100000:0.0:1789760774.102550:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760774.102652:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508428928 00000100:00100000:1.0:1789760774.102663:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508428928 00000100:00100000:1.0:1789760774.102718:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753117130817 time 753165469258 state 0x0 ref 1 00000100:00100000:1.0:1789760774.102732:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753165729674 time 753165469258 state 0x1 ref 1 00000100:00100000:1.0:1789760774.102857:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753165729674 time 753207566488 state 0x0 ref 1 00000100:00100000:1.0:1789760774.102871:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508428928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760774.102919:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23801856, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:1.0:1789760774.102927:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe565f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.102943:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.102947:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.102954:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe565f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.102967:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe565f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760774.139697:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23801856 -> 23805952 00010000:00010000:2.0:1789760774.139716:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe565f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.139733:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe565f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.140012:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973131 is committed 00010000:00010000:2.0:1789760774.140072:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe565f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.140082:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760774.140087:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe565f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.140095:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe565f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.140115:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe565f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.140122:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe565f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23801856->23805951] (req 23801856->23805951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.140229:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508428928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37354us (37682us total) trans 4294973131 rc 0/0 00000100:00100000:2.0:1789760774.140239:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5843 00000100:00100000:2.0:1789760774.140249:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753165729674 time 753207566488 state 0x2 ref 0 00000100:00100000:2.0:1789760774.144878:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760774.144974:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508429056 00000100:00100000:0.0:1789760774.144984:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508429056 00000100:00100000:0.0:1789760774.145044:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753165729674 time 753207566488 state 0x0 ref 1 00000100:00100000:0.0:1789760774.145059:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753207826904 time 753207566488 state 0x1 ref 1 00000100:00100000:0.0:1789760774.145089:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753207826904 time 753249799788 state 0x0 ref 1 00000100:00100000:0.0:1789760774.145103:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508429056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760774.145151:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23805952, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:0.0:1789760774.145161:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.145176:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760774.145180:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.145187:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.145200:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.182721:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973132 is committed 00002000:00010000:0.0:1789760774.183622:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23805952 -> 23810048 00010000:00010000:0.0:1789760774.183639:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.183657:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.183666:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.183672:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760774.183677:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.183686:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.183704:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.183711:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe565fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23805952->23810047] (req 23805952->23810047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.183864:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508429056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38757us (38990us total) trans 4294973132 rc 0/0 00000100:00100000:0.0:1789760774.183875:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5844 00000100:00100000:0.0:1789760774.183884:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753207826904 time 753249799788 state 0x2 ref 0 00000100:00100000:0.0:1789760774.188940:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760774.189038:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508429184 00000100:00100000:2.0:1789760774.189048:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508429184 00000100:00100000:2.0:1789760774.189104:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753207826904 time 753249799788 state 0x0 ref 1 00000100:00100000:2.0:1789760774.189117:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753250060204 time 753249799788 state 0x1 ref 1 00000100:00100000:2.0:1789760774.189147:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753250060204 time 753293857357 state 0x0 ref 1 00000100:00100000:2.0:1789760774.189158:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508429184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760774.189213:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23810048, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:2.0:1789760774.189224:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56602 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.189238:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760774.189243:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.189249:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56602 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.189262:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56602 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.214917:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973133 is committed 00002000:00010000:2.0:1789760774.214955:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23810048 -> 23814144 00010000:00010000:2.0:1789760774.214971:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56602 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.214983:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56602 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.214993:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56602 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.214999:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760774.215004:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56602 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.215013:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56602 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.215257:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56602 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.215265:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56602 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23810048->23814143] (req 23810048->23814143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.215555:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508429184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26392us (26617us total) trans 4294973133 rc 0/0 00000100:00100000:2.0:1789760774.215565:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5845 00000100:00100000:2.0:1789760774.215574:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753250060204 time 753293857357 state 0x2 ref 0 00000100:00100000:0.0:1789760774.222526:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760774.222619:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508429312 00000100:00100000:1.0:1789760774.222629:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508429312 00000100:00100000:1.0:1789760774.222683:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753250060204 time 753293857357 state 0x0 ref 1 00000100:00100000:1.0:1789760774.222696:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753294117773 time 753293857357 state 0x1 ref 1 00000100:00100000:1.0:1789760774.222726:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753294117773 time 753327435834 state 0x0 ref 1 00000100:00100000:1.0:1789760774.222738:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508429312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760774.222789:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23814144, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:1.0:1789760774.222798:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56609 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.222813:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.222817:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.222823:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56609 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.222847:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56609 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.247534:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973134 is committed 00002000:00010000:1.0:1789760774.247613:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23814144 -> 23818240 00010000:00010000:1.0:1789760774.247628:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56609 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.247641:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56609 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.247650:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56609 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.247658:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760774.247663:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56609 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.247672:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56609 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.247692:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56609 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.247700:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56609 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23814144->23818239] (req 23814144->23818239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.247799:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508429312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25058us (25277us total) trans 4294973134 rc 0/0 00000100:00100000:1.0:1789760774.247808:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5846 00000100:00100000:1.0:1789760774.247818:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753294117773 time 753327435834 state 0x2 ref 0 00000100:00100000:1.0:1789760774.255497:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760774.255608:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508429440 00000100:00100000:0.0:1789760774.255618:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508429440 00000100:00100000:0.0:1789760774.255671:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753294117773 time 753327435834 state 0x0 ref 1 00000100:00100000:0.0:1789760774.255687:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753327696250 time 753327435834 state 0x1 ref 1 00000100:00100000:0.0:1789760774.255760:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753327696250 time 753360469215 state 0x0 ref 1 00000100:00100000:0.0:1789760774.255773:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508429440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760774.255818:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23818240, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:0.0:1789760774.255837:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56610 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.255854:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760774.255858:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.255864:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56610 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.255886:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56610 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.289106:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23818240 -> 23822336 00010000:00010000:0.0:1789760774.289126:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56610 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.289145:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56610 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.289153:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56610 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.289161:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760774.289166:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56610 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.289185:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56610 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.289206:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56610 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.289214:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56610 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23818240->23822335] (req 23818240->23822335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.289335:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508429440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33558us (33843us total) trans 4294973135 rc 0/0 00000100:00100000:0.0:1789760774.289345:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5847 00000100:00100000:0.0:1789760774.289355:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753327696250 time 753360469215 state 0x2 ref 0 00000001:00080000:0.0:1789760774.289391:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973135 is committed 00000100:00100000:0.0:1789760774.294347:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760774.294492:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508429568 00000100:00100000:2.0:1789760774.294502:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508429568 00000100:00100000:2.0:1789760774.294557:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753327696250 time 753360469215 state 0x0 ref 1 00000100:00100000:2.0:1789760774.294572:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753360729631 time 753360469215 state 0x1 ref 1 00000100:00100000:2.0:1789760774.294605:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753360729631 time 753399315599 state 0x0 ref 1 00000100:00100000:2.0:1789760774.294617:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508429568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760774.294666:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23822336, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:2.0:1789760774.294676:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56617 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.294692:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760774.294696:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.294703:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56617 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.294717:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56617 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.331247:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973136 is committed 00002000:00010000:1.0:1789760774.331364:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23822336 -> 23826432 00010000:00010000:1.0:1789760774.331378:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56617 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.331391:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56617 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.331401:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56617 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.331412:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760774.331417:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56617 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.331426:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56617 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.331446:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56617 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.331455:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56617 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23822336->23826431] (req 23822336->23826431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.331552:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508429568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36932us (37211us total) trans 4294973136 rc 0/0 00000100:00100000:1.0:1789760774.331562:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5848 00000100:00100000:1.0:1789760774.331572:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753360729631 time 753399315599 state 0x2 ref 0 00000100:00100000:0.0:1789760774.339786:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760774.339920:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508429696 00000100:00100000:1.0:1789760774.339930:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508429696 00000100:00100000:1.0:1789760774.339987:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753360729631 time 753399315599 state 0x0 ref 1 00000100:00100000:1.0:1789760774.340004:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753399576015 time 753399315599 state 0x1 ref 1 00000100:00100000:1.0:1789760774.340034:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753399576015 time 753444745278 state 0x0 ref 1 00000100:00100000:1.0:1789760774.340050:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508429696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760774.340097:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23826432, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:1.0:1789760774.340117:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5661e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.340133:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.340137:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.340144:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5661e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.340158:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5661e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.367225:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973137 is committed 00002000:00010000:3.0:1789760774.367476:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23826432 -> 23830528 00010000:00010000:3.0:1789760774.367490:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5661e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.367504:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5661e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.367512:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5661e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.367519:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760774.367523:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5661e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.367529:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5661e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.367547:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5661e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.367552:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe5661e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23826432->23830527] (req 23826432->23830527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.367630:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508429696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27580us (27851us total) trans 4294973137 rc 0/0 00000100:00100000:3.0:1789760774.367637:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5849 00000100:00100000:3.0:1789760774.367645:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753399576015 time 753444745278 state 0x2 ref 0 00000100:00100000:2.0:1789760774.371521:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760774.371586:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508429824 00000100:00100000:0.0:1789760774.371595:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508429824 00000100:00100000:0.0:1789760774.371637:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753399576015 time 753444745278 state 0x0 ref 1 00000100:00100000:0.0:1789760774.371648:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753445005694 time 753444745278 state 0x1 ref 1 00000100:00100000:0.0:1789760774.371678:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753445005694 time 753476388041 state 0x0 ref 1 00000100:00100000:0.0:1789760774.371691:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508429824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760774.372017:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23830528, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:0.0:1789760774.372026:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56625 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.372041:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760774.372045:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.372050:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56625 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.372104:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56625 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.402808:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973138 is committed 00002000:00010000:0.0:1789760774.402988:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23830528 -> 23834624 00010000:00010000:0.0:1789760774.403002:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56625 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.403022:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56625 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.403030:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56625 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.403038:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760774.403042:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56625 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.403051:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56625 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.403071:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56625 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.403079:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56625 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23830528->23834623] (req 23830528->23834623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.403279:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508429824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31585us (31760us total) trans 4294973138 rc 0/0 00000100:00100000:0.0:1789760774.403331:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5850 00000100:00100000:0.0:1789760774.403341:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753445005694 time 753476388041 state 0x2 ref 0 00000100:00100000:0.0:1789760774.408942:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760774.409317:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508429952 00000100:00100000:1.0:1789760774.409325:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508429952 00000100:00100000:1.0:1789760774.409376:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753445005694 time 753476388041 state 0x0 ref 1 00000100:00100000:1.0:1789760774.409399:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753476648457 time 753476388041 state 0x1 ref 1 00000100:00100000:1.0:1789760774.409425:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753476648457 time 753514136093 state 0x0 ref 1 00000100:00100000:1.0:1789760774.409436:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508429952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760774.409484:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23834624, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:1.0:1789760774.409492:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5662c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.409508:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.409512:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.409518:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5662c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.409531:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5662c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760774.436337:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23834624 -> 23838720 00010000:00010000:0.0:1789760774.436356:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5662c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.436369:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5662c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.436378:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5662c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.436386:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760774.436391:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5662c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.436399:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5662c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.436418:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5662c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.436426:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5662c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23834624->23838719] (req 23834624->23838719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.436509:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508429952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27069us (27572us total) trans 4294973139 rc 0/0 00000100:00100000:0.0:1789760774.436519:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5851 00000100:00100000:0.0:1789760774.436529:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753476648457 time 753514136093 state 0x2 ref 0 00000001:00080000:1.0:1789760774.436677:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973139 is committed 00000100:00100000:2.0:1789760774.440565:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760774.440735:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508430080 00000100:00100000:3.0:1789760774.440798:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508430080 00000100:00100000:3.0:1789760774.440864:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753476648457 time 753514136093 state 0x0 ref 1 00000100:00100000:3.0:1789760774.440880:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753514396509 time 753514136093 state 0x1 ref 1 00000100:00100000:3.0:1789760774.440905:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753514396509 time 753545615517 state 0x0 ref 1 00000100:00100000:3.0:1789760774.440917:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508430080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760774.440965:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23838720, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:3.0:1789760774.440974:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56633 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.440990:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:3.0:1789760774.440995:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.441002:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56633 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.441016:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56633 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.479748:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973140 is committed 00002000:00010000:2.0:1789760774.479875:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23838720 -> 23842816 00010000:00010000:2.0:1789760774.479889:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56633 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.479901:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56633 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.479909:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56633 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.479917:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760774.479922:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56633 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.479930:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56633 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.479949:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56633 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.479956:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56633 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23838720->23842815] (req 23838720->23842815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.480054:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508430080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39133us (39492us total) trans 4294973140 rc 0/0 00000100:00100000:2.0:1789760774.480065:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5852 00000100:00100000:2.0:1789760774.480075:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753514396509 time 753545615517 state 0x2 ref 0 00000100:00100000:2.0:1789760774.487989:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760774.488156:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508430208 00000100:00100000:0.0:1789760774.488166:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508430208 00000100:00100000:0.0:1789760774.488217:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753514396509 time 753545615517 state 0x0 ref 1 00000100:00100000:0.0:1789760774.488232:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753545875933 time 753545615517 state 0x1 ref 1 00000100:00100000:0.0:1789760774.488270:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753545875933 time 753592980212 state 0x0 ref 1 00000100:00100000:0.0:1789760774.488282:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508430208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760774.488359:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23842816, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0xb807 00010000:00010000:0.0:1789760774.488369:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5663a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.488383:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760774.488387:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.488393:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5663a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.488406:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5663a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.517719:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973141 is committed 00002000:00010000:0.0:1789760774.517899:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23842816 -> 23846912 00002000:00010000:0.0:1789760774.517904:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760774 -> 1789760775 00002000:00010000:0.0:1789760774.517906:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760774 -> 1789760775 00010000:00010000:0.0:1789760774.517919:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5663a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.517933:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5663a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.517942:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5663a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.517950:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760774.517955:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5663a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.517964:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5663a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.517987:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5663a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.517995:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5663a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23842816->23846911] (req 23842816->23846911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.518093:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508430208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29807us (30108us total) trans 4294973141 rc 0/0 00000100:00100000:0.0:1789760774.518102:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5853 00000100:00100000:0.0:1789760774.518112:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753545875933 time 753592980212 state 0x2 ref 0 00000100:00100000:0.0:1789760774.522186:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760774.522278:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508430336 00000100:00100000:1.0:1789760774.522321:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508430336 00000100:00100000:1.0:1789760774.522378:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753545875933 time 753592980212 state 0x0 ref 1 00000100:00100000:1.0:1789760774.522394:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753593240628 time 753592980212 state 0x1 ref 1 00000100:00100000:1.0:1789760774.522425:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753593240628 time 753627136039 state 0x0 ref 1 00000100:00100000:1.0:1789760774.522439:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508430336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760774.522488:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23846912, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:1.0:1789760774.522497:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56641 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.522511:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.522516:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.522523:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56641 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.522538:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56641 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.552667:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973142 is committed 00002000:00010000:0.0:1789760774.553084:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23846912 -> 23851008 00010000:00010000:0.0:1789760774.553101:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56641 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.553114:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56641 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.553123:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56641 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.553131:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760774.553137:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56641 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.553148:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56641 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.553170:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56641 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.553189:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56641 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23846912->23851007] (req 23846912->23851007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.553281:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508430336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30840us (31108us total) trans 4294973142 rc 0/0 00000100:00100000:0.0:1789760774.553324:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5854 00000100:00100000:0.0:1789760774.553334:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753593240628 time 753627136039 state 0x2 ref 0 00000100:00100000:2.0:1789760774.557080:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760774.557238:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508430464 00000100:00100000:0.0:1789760774.557249:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508430464 00000100:00100000:0.0:1789760774.557457:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753593240628 time 753627136039 state 0x0 ref 1 00000100:00100000:0.0:1789760774.557472:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753627396455 time 753627136039 state 0x1 ref 1 00000100:00100000:0.0:1789760774.557531:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753627396455 time 753662240403 state 0x0 ref 1 00000100:00100000:0.0:1789760774.557544:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508430464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760774.557591:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23851008, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:0.0:1789760774.557601:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56648 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.557616:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760774.557620:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.557627:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56648 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.557641:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56648 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.597952:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973143 is committed 00002000:00010000:1.0:1789760774.598063:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23851008 -> 23855104 00010000:00010000:1.0:1789760774.598078:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56648 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.598090:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56648 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.598097:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56648 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.598104:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760774.598108:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56648 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.598116:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56648 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.598204:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56648 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.598212:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56648 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23851008->23855103] (req 23851008->23855103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.598341:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508430464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40794us (41265us total) trans 4294973143 rc 0/0 00000100:00100000:1.0:1789760774.598351:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5855 00000100:00100000:1.0:1789760774.598362:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753627396455 time 753662240403 state 0x2 ref 0 00000100:00100000:2.0:1789760774.602461:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760774.602669:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508430592 00000100:00100000:0.0:1789760774.602679:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508430592 00000100:00100000:0.0:1789760774.602729:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753627396455 time 753662240403 state 0x0 ref 1 00000100:00100000:0.0:1789760774.602744:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753662500819 time 753662240403 state 0x1 ref 1 00000100:00100000:0.0:1789760774.602784:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753662500819 time 753707486549 state 0x0 ref 1 00000100:00100000:0.0:1789760774.602797:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508430592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760774.602850:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23855104, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:0.0:1789760774.602860:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5664f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.602875:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760774.602879:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.602885:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5664f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.602899:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5664f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.635550:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973144 is committed 00002000:00010000:1.0:1789760774.635654:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23855104 -> 23859200 00010000:00010000:1.0:1789760774.635667:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5664f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.635679:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5664f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.635688:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5664f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.635696:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760774.635700:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5664f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.635710:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5664f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.635730:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5664f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.635743:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5664f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23855104->23859199] (req 23855104->23859199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.635908:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508430592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33108us (33449us total) trans 4294973144 rc 0/0 00000100:00100000:1.0:1789760774.635919:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5856 00000100:00100000:1.0:1789760774.635928:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753662500819 time 753707486549 state 0x2 ref 0 00000100:00100000:2.0:1789760774.644158:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760774.644404:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508430720 00000100:00100000:0.0:1789760774.644414:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508430720 00000100:00100000:0.0:1789760774.644477:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753662500819 time 753707486549 state 0x0 ref 1 00000100:00100000:0.0:1789760774.644493:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753707746965 time 753707486549 state 0x1 ref 1 00000100:00100000:0.0:1789760774.644576:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753707746965 time 753749286055 state 0x0 ref 1 00000100:00100000:0.0:1789760774.644590:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508430720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760774.644638:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23859200, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:0.0:1789760774.644647:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56656 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.644662:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:0.0:1789760774.644666:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.644671:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56656 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.644685:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56656 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.686587:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973145 is committed 00002000:00010000:1.0:1789760774.686619:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23859200 -> 23863296 00010000:00010000:1.0:1789760774.686639:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56656 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.686652:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56656 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.686662:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56656 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.686670:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760774.686676:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56656 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.686686:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56656 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.686723:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56656 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.686733:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56656 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23859200->23863295] (req 23859200->23863295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.686904:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508430720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42312us (42751us total) trans 4294973145 rc 0/0 00000100:00100000:1.0:1789760774.686914:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5857 00000100:00100000:1.0:1789760774.686924:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753707746965 time 753749286055 state 0x2 ref 0 00000100:00100000:3.0:1789760774.692397:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760774.692569:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508430848 00000100:00100000:1.0:1789760774.692579:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508430848 00000100:00100000:1.0:1789760774.692634:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753707746965 time 753749286055 state 0x0 ref 1 00000100:00100000:1.0:1789760774.692648:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753749546471 time 753749286055 state 0x1 ref 1 00000100:00100000:1.0:1789760774.692732:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753749546471 time 753797441206 state 0x0 ref 1 00000100:00100000:1.0:1789760774.692848:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508430848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760774.692897:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23863296, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:1.0:1789760774.692906:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5665d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.692920:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.692925:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.692932:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5665d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.692946:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5665d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.723306:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973146 is committed 00002000:00010000:2.0:1789760774.723492:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23863296 -> 23867392 00010000:00010000:2.0:1789760774.723507:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5665d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.723520:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5665d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.723528:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5665d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.723535:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760774.723540:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5665d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.723549:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5665d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.723569:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5665d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.723575:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5665d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23863296->23867391] (req 23863296->23867391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.723696:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508430848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30846us (31303us total) trans 4294973146 rc 0/0 00000100:00100000:2.0:1789760774.723708:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5858 00000100:00100000:2.0:1789760774.723718:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753749546471 time 753797441206 state 0x2 ref 0 00000100:00100000:2.0:1789760774.732482:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760774.732607:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508430976 00000100:00100000:1.0:1789760774.732618:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508430976 00000100:00100000:1.0:1789760774.732746:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753749546471 time 753797441206 state 0x0 ref 1 00000100:00100000:1.0:1789760774.732761:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753797701622 time 753797441206 state 0x1 ref 1 00000100:00100000:1.0:1789760774.732909:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753797701622 time 753837618741 state 0x0 ref 1 00000100:00100000:1.0:1789760774.732923:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508430976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760774.732972:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23867392, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:1.0:1789760774.732981:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56664 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.732997:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.733001:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.733008:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56664 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.733022:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56664 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.778730:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973147 is committed 00002000:00010000:1.0:1789760774.778843:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23867392 -> 23871488 00010000:00010000:1.0:1789760774.778858:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56664 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.778870:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56664 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.778879:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56664 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.778886:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760774.778891:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56664 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.778899:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56664 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.778917:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56664 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.778925:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56664 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23867392->23871487] (req 23867392->23871487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.779017:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508430976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46092us (46541us total) trans 4294973147 rc 0/0 00000100:00100000:1.0:1789760774.779031:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5859 00000100:00100000:1.0:1789760774.779040:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753797701622 time 753837618741 state 0x2 ref 0 00000100:00100000:0.0:1789760774.783479:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760774.783559:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508431104 00000100:00100000:1.0:1789760774.783569:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508431104 00000100:00100000:1.0:1789760774.783627:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753797701622 time 753837618741 state 0x0 ref 1 00000100:00100000:1.0:1789760774.783643:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753837879157 time 753837618741 state 0x1 ref 1 00000100:00100000:1.0:1789760774.783671:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753837879157 time 753888382154 state 0x0 ref 1 00000100:00100000:1.0:1789760774.783684:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508431104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760774.783731:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23871488, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:1.0:1789760774.783740:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5666b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.783754:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.783758:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.783764:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5666b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.783778:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5666b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.819637:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973148 is committed 00002000:00010000:0.0:1789760774.819739:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23871488 -> 23875584 00010000:00010000:0.0:1789760774.819755:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5666b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.819769:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5666b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.819777:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5666b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.819784:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760774.819789:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5666b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.819797:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5666b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.819815:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5666b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.819822:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5666b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23871488->23875583] (req 23871488->23875583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.819920:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508431104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36233us (36582us total) trans 4294973148 rc 0/0 00000100:00100000:0.0:1789760774.819929:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5860 00000100:00100000:0.0:1789760774.819939:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753837879157 time 753888382154 state 0x2 ref 0 00000100:00100000:0.0:1789760774.825418:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760774.825557:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508431232 00000100:00100000:2.0:1789760774.825568:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508431232 00000100:00100000:2.0:1789760774.825623:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753837879157 time 753888382154 state 0x0 ref 1 00000100:00100000:2.0:1789760774.825639:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753888642570 time 753888382154 state 0x1 ref 1 00000100:00100000:2.0:1789760774.825670:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753888642570 time 753930380637 state 0x0 ref 1 00000100:00100000:2.0:1789760774.825683:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508431232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760774.825731:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23875584, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:2.0:1789760774.825741:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56672 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.825757:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:2.0:1789760774.825762:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.825767:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56672 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.825782:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56672 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.860881:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973149 is committed 00002000:00010000:2.0:1789760774.860938:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23875584 -> 23879680 00010000:00010000:2.0:1789760774.860962:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56672 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.860980:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56672 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.860988:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56672 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.860995:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760774.861000:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56672 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.861009:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56672 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.861029:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56672 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.861036:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56672 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23875584->23879679] (req 23875584->23879679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.861122:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508431232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35437us (35710us total) trans 4294973149 rc 0/0 00000100:00100000:2.0:1789760774.861132:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5861 00000100:00100000:2.0:1789760774.861142:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753888642570 time 753930380637 state 0x2 ref 0 00000100:00100000:2.0:1789760774.864917:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760774.865142:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508431360 00000100:00100000:1.0:1789760774.865153:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508431360 00000100:00100000:1.0:1789760774.865209:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753888642570 time 753930380637 state 0x0 ref 1 00000100:00100000:1.0:1789760774.865224:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753930641053 time 753930380637 state 0x1 ref 1 00000100:00100000:1.0:1789760774.865260:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753930641053 time 753969971182 state 0x0 ref 1 00000100:00100000:1.0:1789760774.865272:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508431360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760774.865418:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23879680, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:1.0:1789760774.865427:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56679 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.865443:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=753s v=(0 0) 00010000:00010000:1.0:1789760774.865447:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.865453:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56679 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.865466:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56679 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.896841:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973150 is committed 00002000:00010000:0.0:1789760774.897022:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23879680 -> 23883776 00010000:00010000:0.0:1789760774.897038:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56679 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897051:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56679 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897060:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56679 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897067:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760774.897072:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56679 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897086:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56679 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897107:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56679 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.897116:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56679 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23879680->23883775] (req 23879680->23883775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.897209:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508431360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31934us (32296us total) trans 4294973150 rc 0/0 00000100:00100000:0.0:1789760774.897220:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5862 00000100:00100000:0.0:1789760774.897231:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753930641053 time 753969971182 state 0x2 ref 0 00000100:00100000:0.0:1789760774.902077:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760774.902206:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508431488 00000100:00100000:1.0:1789760774.902216:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508431488 00000100:00100000:1.0:1789760774.902275:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753930641053 time 753969971182 state 0x0 ref 1 00000100:00100000:1.0:1789760774.902319:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753970231598 time 753969971182 state 0x1 ref 1 00000100:00100000:1.0:1789760774.902348:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753970231598 time 754007058755 state 0x0 ref 1 00000100:00100000:1.0:1789760774.902364:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508431488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760774.902413:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23883776, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:1.0:1789760774.902422:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56680 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.902437:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760774.902441:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.902448:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56680 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.902462:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56680 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.915388:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760774.958883:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23883776 -> 23887872 00010000:00010000:2.0:1789760774.958904:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56680 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.958916:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56680 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.958924:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56680 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.958930:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760774.958935:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56680 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.958943:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56680 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.958962:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56680 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.958969:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56680 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23883776->23887871] (req 23883776->23887871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.959062:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508431488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56695us (56991us total) trans 4294973151 rc 0/0 00000100:00100000:2.0:1789760774.959073:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5863 00000100:00100000:2.0:1789760774.959083:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753970231598 time 754007058755 state 0x2 ref 0 00000001:00080000:3.0:1789760774.959161:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973151 is committed 00000100:00100000:3.0:1789760774.962872:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760774.963003:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508431616 00000100:00100000:2.0:1789760774.963014:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508431616 00000100:00100000:2.0:1789760774.963073:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 753970231598 time 754007058755 state 0x0 ref 1 00000100:00100000:2.0:1789760774.963089:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754007319171 time 754007058755 state 0x1 ref 1 00000100:00100000:2.0:1789760774.963175:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754007319171 time 754067884860 state 0x0 ref 1 00000100:00100000:2.0:1789760774.963188:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508431616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760774.963238:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23887872, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:2.0:1789760774.963247:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56687 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.963263:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760774.963268:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.963275:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56687 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.963340:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56687 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760774.990196:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23887872 -> 23891968 00010000:00010000:2.0:1789760774.990215:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56687 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.990228:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56687 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.990237:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56687 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.990239:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973152 is committed 00010000:00010000:2.0:1789760774.990248:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760774.990253:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56687 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.990265:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56687 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.990328:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56687 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.990337:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56687 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23887872->23891967] (req 23887872->23891967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.990434:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508431616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27244us (27568us total) trans 4294973152 rc 0/0 00000100:00100000:2.0:1789760774.990455:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5864 00000100:00100000:2.0:1789760774.990465:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754007319171 time 754067884860 state 0x2 ref 0 00000100:00100000:1.0:1789760774.994493:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760774.994576:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508431744 00000100:00100000:0.0:1789760774.994587:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508431744 00000100:00100000:0.0:1789760774.994641:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754007319171 time 754067884860 state 0x0 ref 1 00000100:00100000:0.0:1789760774.994656:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754068145276 time 754067884860 state 0x1 ref 1 00000100:00100000:0.0:1789760774.994692:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754068145276 time 754099401890 state 0x0 ref 1 00000100:00100000:0.0:1789760774.994705:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508431744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760774.994751:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23891968, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:0.0:1789760774.994760:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5668e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.994777:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760774.994782:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.994789:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5668e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.994803:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5668e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.025536:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973153 is committed 00002000:00010000:3.0:1789760775.025578:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23891968 -> 23896064 00010000:00010000:3.0:1789760775.025593:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5668e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.025606:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5668e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.025615:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5668e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.025623:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760775.025628:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5668e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.025638:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5668e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.025659:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5668e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.025667:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5668e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23891968->23896063] (req 23891968->23896063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.025959:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508431744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31252us (31471us total) trans 4294973153 rc 0/0 00000100:00100000:3.0:1789760775.025970:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5865 00000100:00100000:3.0:1789760775.025979:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754068145276 time 754099401890 state 0x2 ref 0 00000100:00100000:0.0:1789760775.030936:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760775.031016:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508431872 00000100:00100000:2.0:1789760775.031028:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508431872 00000100:00100000:2.0:1789760775.031103:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754068145276 time 754099401890 state 0x0 ref 1 00000100:00100000:2.0:1789760775.031130:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754099662306 time 754099401890 state 0x1 ref 1 00000100:00100000:2.0:1789760775.031166:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754099662306 time 754135876371 state 0x0 ref 1 00000100:00100000:2.0:1789760775.031179:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508431872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760775.031228:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23896064, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:2.0:1789760775.031237:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56695 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760775.031240:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789760775.031255:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760775.031260:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.031266:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56695 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.031280:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56695 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.038629:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887e67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534084352:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760775.038701:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760775.038776:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534084352 00000100:00100000:0.0:1789760775.038783:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534084352 00000100:00100000:0.0:1789760775.038913:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 457 00000100:00100000:0.0:1789760775.038923:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534084352:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760775.038962:0:7358:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789760775.039009:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534084352:12345-0@lo:41:osp-pre-1-0.0 Request processed in 87us (307us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760775.039016:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 457 00000100:00100000:0.0:1789760775.039061:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887e67b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534084352:0@lo:41:osp-pre-1-0.0 00000001:00080000:0.0:1789760775.065891:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973154 is committed 00002000:00010000:3.0:1789760775.066087:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23896064 -> 23900160 00010000:00010000:3.0:1789760775.066102:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56695 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.066114:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56695 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.066123:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56695 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.066130:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760775.066135:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56695 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.066150:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56695 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.066169:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56695 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.066186:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56695 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23896064->23900159] (req 23896064->23900159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.066281:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508431872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35097us (35349us total) trans 4294973154 rc 0/0 00000100:00100000:3.0:1789760775.066575:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5866 00000100:00100000:3.0:1789760775.066586:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754099662306 time 754135876371 state 0x2 ref 0 00000100:00100000:0.0:1789760775.070715:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760775.071426:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508432000 00000100:00100000:2.0:1789760775.071437:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508432000 00000100:00100000:2.0:1789760775.071490:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754099662306 time 754135876371 state 0x0 ref 1 00000100:00100000:2.0:1789760775.071504:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754136136787 time 754135876371 state 0x1 ref 1 00000100:00100000:2.0:1789760775.071528:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754136136787 time 754176238647 state 0x0 ref 1 00000100:00100000:2.0:1789760775.071540:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508432000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760775.071588:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23900160, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:2.0:1789760775.071596:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5669c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.071612:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760775.071616:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.071621:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5669c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.071634:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5669c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.104959:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23900160 -> 23904256 00010000:00010000:1.0:1789760775.104977:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5669c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.104978:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973155 is committed 00010000:00010000:1.0:1789760775.104991:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5669c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.105000:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5669c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.105008:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760775.105013:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5669c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.105023:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5669c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.105042:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5669c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.105050:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5669c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23900160->23904255] (req 23900160->23904255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.105140:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508432000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33597us (34428us total) trans 4294973155 rc 0/0 00000100:00100000:1.0:1789760775.105149:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5867 00000100:00100000:1.0:1789760775.105159:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754136136787 time 754176238647 state 0x2 ref 0 00000100:00100000:0.0:1789760775.110732:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760775.110847:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508432128 00000100:00100000:3.0:1789760775.110856:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508432128 00000100:00100000:3.0:1789760775.110906:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754136136787 time 754176238647 state 0x0 ref 1 00000100:00100000:3.0:1789760775.110921:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754176499063 time 754176238647 state 0x1 ref 1 00000100:00100000:3.0:1789760775.110990:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754176499063 time 754215700211 state 0x0 ref 1 00000100:00100000:3.0:1789760775.111004:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508432128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760775.111059:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23904256, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:3.0:1789760775.111068:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe566a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.111084:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.111089:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.111095:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe566a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.111108:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe566a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.145762:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973156 is committed 00002000:00010000:0.0:1789760775.145876:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23904256 -> 23908352 00010000:00010000:0.0:1789760775.145891:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe566a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.145905:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe566a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.145913:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe566a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.145921:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760775.145926:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe566a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.145934:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe566a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.145953:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe566a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.145960:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe566a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23904256->23908351] (req 23904256->23908351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.146102:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508432128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35096us (35374us total) trans 4294973156 rc 0/0 00000100:00100000:0.0:1789760775.146112:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5868 00000100:00100000:0.0:1789760775.146123:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754176499063 time 754215700211 state 0x2 ref 0 00000100:00100000:0.0:1789760775.150795:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760775.151269:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508432256 00000100:00100000:2.0:1789760775.151279:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508432256 00000100:00100000:2.0:1789760775.151404:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754176499063 time 754215700211 state 0x0 ref 1 00000100:00100000:2.0:1789760775.151420:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754215960627 time 754215700211 state 0x1 ref 1 00000100:00100000:2.0:1789760775.151495:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754215960627 time 754256203817 state 0x0 ref 1 00000100:00100000:2.0:1789760775.151509:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508432256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760775.151558:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23908352, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:2.0:1789760775.151568:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe566aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.151583:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:2.0:1789760775.151587:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.151593:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe566aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.151607:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe566aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.182908:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973157 is committed 00002000:00010000:3.0:1789760775.182996:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23908352 -> 23912448 00010000:00010000:3.0:1789760775.183012:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe566aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.183022:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe566aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.183031:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe566aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.183038:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760775.183042:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe566aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.183051:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe566aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.183070:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe566aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.183077:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe566aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23908352->23912447] (req 23908352->23912447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.183433:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508432256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31921us (32642us total) trans 4294973157 rc 0/0 00000100:00100000:3.0:1789760775.183443:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5869 00000100:00100000:3.0:1789760775.183453:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754215960627 time 754256203817 state 0x2 ref 0 00000100:00100000:3.0:1789760775.192033:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760775.192108:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508432384 00000100:00100000:1.0:1789760775.192118:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508432384 00000100:00100000:1.0:1789760775.192171:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754215960627 time 754256203817 state 0x0 ref 1 00000100:00100000:1.0:1789760775.192248:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754256464233 time 754256203817 state 0x1 ref 1 00000100:00100000:1.0:1789760775.192380:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754256464233 time 754297089718 state 0x0 ref 1 00000100:00100000:1.0:1789760775.192392:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508432384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760775.192446:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23912448, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:1.0:1789760775.192455:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.192470:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.192474:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.192480:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.192494:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.228549:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23912448 -> 23916544 00010000:00010000:1.0:1789760775.228570:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.228573:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973158 is committed 00010000:00010000:1.0:1789760775.228582:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.228592:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.228599:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760775.228604:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.228614:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.228634:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.228643:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23912448->23916543] (req 23912448->23916543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.228764:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508432384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36369us (36736us total) trans 4294973158 rc 0/0 00000100:00100000:1.0:1789760775.228775:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5870 00000100:00100000:1.0:1789760775.228785:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754256464233 time 754297089718 state 0x2 ref 0 00000100:00100000:0.0:1789760775.234956:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760775.235242:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508432512 00000100:00100000:3.0:1789760775.235251:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508432512 00000100:00100000:3.0:1789760775.235332:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754256464233 time 754297089718 state 0x0 ref 1 00000100:00100000:3.0:1789760775.235348:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754297350134 time 754297089718 state 0x1 ref 1 00000100:00100000:3.0:1789760775.235363:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754297350134 time 754340073692 state 0x0 ref 1 00000100:00100000:3.0:1789760775.235374:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508432512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760775.235421:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23916544, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:3.0:1789760775.235432:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.235447:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.235452:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.235459:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.235472:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.276690:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23916544 -> 23920640 00010000:00010000:3.0:1789760775.276714:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.276727:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.276735:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.276743:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760775.276748:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.276757:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.276776:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.276783:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23916544->23920639] (req 23916544->23920639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.276956:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508432512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41578us (42002us total) trans 4294973159 rc 0/0 00000100:00100000:3.0:1789760775.276967:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5871 00000100:00100000:3.0:1789760775.276977:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754297350134 time 754340073692 state 0x2 ref 0 00000001:00080000:1.0:1789760775.277051:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973159 is committed 00000100:00100000:0.0:1789760775.281634:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760775.281712:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508432640 00000100:00100000:3.0:1789760775.281722:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508432640 00000100:00100000:3.0:1789760775.281774:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754297350134 time 754340073692 state 0x0 ref 1 00000100:00100000:3.0:1789760775.281800:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754340334108 time 754340073692 state 0x1 ref 1 00000100:00100000:3.0:1789760775.281843:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754340334108 time 754386553175 state 0x0 ref 1 00000100:00100000:3.0:1789760775.281855:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508432640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760775.281901:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23920640, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:3.0:1789760775.281910:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe566bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.281924:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.281928:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.281934:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe566bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.281947:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe566bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.322139:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23920640 -> 23924736 00010000:00010000:0.0:1789760775.322160:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe566bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.322184:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe566bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.322194:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe566bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.322201:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760775.322205:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe566bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.322213:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe566bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.322238:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe566bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.322246:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe566bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23920640->23924735] (req 23920640->23924735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.322420:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508432640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40562us (40791us total) trans 4294973160 rc 0/0 00000100:00100000:0.0:1789760775.322430:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5872 00000100:00100000:0.0:1789760775.322441:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754340334108 time 754386553175 state 0x2 ref 0 00000001:00080000:1.0:1789760775.323142:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973160 is committed 00000100:00100000:0.0:1789760775.326723:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760775.326921:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508432768 00000100:00100000:1.0:1789760775.326930:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508432768 00000100:00100000:1.0:1789760775.326981:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754340334108 time 754386553175 state 0x0 ref 1 00000100:00100000:1.0:1789760775.326996:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754386813591 time 754386553175 state 0x1 ref 1 00000100:00100000:1.0:1789760775.327056:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754386813591 time 754431766251 state 0x0 ref 1 00000100:00100000:1.0:1789760775.327068:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508432768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760775.327114:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23924736, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:1.0:1789760775.327122:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.327139:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.327144:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.327151:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.327164:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.359065:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973161 is committed 00002000:00010000:1.0:1789760775.359151:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23924736 -> 23928832 00010000:00010000:1.0:1789760775.359167:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.359180:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.359189:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.359196:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760775.359201:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.359209:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.359229:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.359238:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23924736->23928831] (req 23924736->23928831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.359404:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508432768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32325us (32677us total) trans 4294973161 rc 0/0 00000100:00100000:1.0:1789760775.359416:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5873 00000100:00100000:1.0:1789760775.359426:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754386813591 time 754431766251 state 0x2 ref 0 00000100:00100000:0.0:1789760775.365891:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760775.366112:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508432896 00000100:00100000:3.0:1789760775.366123:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508432896 00000100:00100000:3.0:1789760775.366185:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754386813591 time 754431766251 state 0x0 ref 1 00000100:00100000:3.0:1789760775.366200:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754432026667 time 754431766251 state 0x1 ref 1 00000100:00100000:3.0:1789760775.366227:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754432026667 time 754470938075 state 0x0 ref 1 00000100:00100000:3.0:1789760775.366240:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508432896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760775.366317:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23928832, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:3.0:1789760775.366328:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.366343:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.366348:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.366355:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.366369:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.411644:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23928832 -> 23932928 00010000:00010000:3.0:1789760775.411665:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.411685:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.411694:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.411701:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760775.411706:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.411715:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.411735:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.411933:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973162 is committed 00010000:00010000:3.0:1789760775.412043:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23928832->23932927] (req 23928832->23932927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.412986:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508432896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46740us (47159us total) trans 4294973162 rc 0/0 00000100:00100000:3.0:1789760775.412999:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5874 00000100:00100000:3.0:1789760775.413010:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754432026667 time 754470938075 state 0x2 ref 0 00000100:00100000:0.0:1789760775.418388:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760775.418537:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508433024 00000100:00100000:3.0:1789760775.418547:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508433024 00000100:00100000:3.0:1789760775.418605:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754432026667 time 754470938075 state 0x0 ref 1 00000100:00100000:3.0:1789760775.418619:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754471198491 time 754470938075 state 0x1 ref 1 00000100:00100000:3.0:1789760775.418638:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754471198491 time 754523349372 state 0x0 ref 1 00000100:00100000:3.0:1789760775.418649:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508433024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760775.418707:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23932928, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:3.0:1789760775.418717:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.418745:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.418750:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.418757:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.418772:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.452539:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973163 is committed 00002000:00010000:3.0:1789760775.452684:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23932928 -> 23937024 00010000:00010000:3.0:1789760775.452699:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.452712:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.452726:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.452734:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760775.452739:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.452749:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.452768:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.452775:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23932928->23937023] (req 23932928->23937023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.452903:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508433024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34251us (34519us total) trans 4294973163 rc 0/0 00000100:00100000:3.0:1789760775.452914:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5875 00000100:00100000:3.0:1789760775.452923:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754471198491 time 754523349372 state 0x2 ref 0 00000100:00100000:0.0:1789760775.462259:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760775.462452:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508433152 00000100:00100000:1.0:1789760775.462462:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508433152 00000100:00100000:1.0:1789760775.462516:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754471198491 time 754523349372 state 0x0 ref 1 00000100:00100000:1.0:1789760775.462531:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754523609788 time 754523349372 state 0x1 ref 1 00000100:00100000:1.0:1789760775.462564:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754523609788 time 754567274312 state 0x0 ref 1 00000100:00100000:1.0:1789760775.462577:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508433152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760775.462704:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23937024, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:1.0:1789760775.462713:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.462728:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.462733:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.462739:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.462757:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.503695:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973164 is committed 00002000:00010000:0.0:1789760775.503744:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23937024 -> 23941120 00010000:00010000:0.0:1789760775.503760:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.503774:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.503783:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.503790:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760775.503796:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.503805:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.503834:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.503843:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe566db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23937024->23941119] (req 23937024->23941119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.503985:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508433152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41405us (41730us total) trans 4294973164 rc 0/0 00000100:00100000:0.0:1789760775.503994:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5876 00000100:00100000:0.0:1789760775.504005:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754523609788 time 754567274312 state 0x2 ref 0 00000100:00100000:0.0:1789760775.510995:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760775.511081:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508433280 00000100:00100000:3.0:1789760775.511092:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508433280 00000100:00100000:3.0:1789760775.511143:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754523609788 time 754567274312 state 0x0 ref 1 00000100:00100000:3.0:1789760775.511159:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754567534728 time 754567274312 state 0x1 ref 1 00000100:00100000:3.0:1789760775.511200:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754567534728 time 754615910121 state 0x0 ref 1 00000100:00100000:3.0:1789760775.511213:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508433280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760775.511258:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23941120, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0xb807 00010000:00010000:3.0:1789760775.511264:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.511280:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.511302:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.511309:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.511323:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.572745:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973165 is committed 00002000:00010000:3.0:1789760775.572785:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23941120 -> 23945216 00002000:00010000:3.0:1789760775.572789:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760775 -> 1789760776 00002000:00010000:3.0:1789760775.572792:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760775 -> 1789760776 00010000:00010000:3.0:1789760775.572804:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.572817:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.572837:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.572844:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760775.572850:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.572858:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.572878:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.572885:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23941120->23945215] (req 23941120->23945215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.572980:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508433280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61764us (61990us total) trans 4294973165 rc 0/0 00000100:00100000:3.0:1789760775.572990:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5877 00000100:00100000:3.0:1789760775.573001:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754567534728 time 754615910121 state 0x2 ref 0 00000100:00100000:2.0:1789760775.578405:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760775.578697:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508433408 00000100:00100000:3.0:1789760775.578707:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508433408 00000100:00100000:3.0:1789760775.579280:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754567534728 time 754615910121 state 0x0 ref 1 00000100:00100000:3.0:1789760775.579327:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754616170537 time 754615910121 state 0x1 ref 1 00000100:00100000:0.0:1789760775.579456:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754616170537 time 754684164304 state 0x0 ref 1 00000100:00100000:0.0:1789760775.579473:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508433408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760775.579527:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23945216, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:0.0:1789760775.579536:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe566e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.579551:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:0.0:1789760775.579556:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.579563:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe566e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.579577:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe566e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760775.611695:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23945216 -> 23949312 00010000:00010000:0.0:1789760775.611719:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe566e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.611733:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe566e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.611743:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe566e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.611750:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760775.611755:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe566e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.611764:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe566e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.611784:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe566e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.611793:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe566e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23945216->23949311] (req 23945216->23949311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.611890:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508433408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32417us (33490us total) trans 4294973166 rc 0/0 00000100:00100000:0.0:1789760775.611901:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5878 00000100:00100000:0.0:1789760775.611911:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754616170537 time 754684164304 state 0x2 ref 0 00000001:00080000:3.0:1789760775.614069:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973166 is committed 00000100:00100000:0.0:1789760775.617232:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760775.617432:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508433536 00000100:00100000:1.0:1789760775.617442:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508433536 00000100:00100000:1.0:1789760775.617499:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754616170537 time 754684164304 state 0x0 ref 1 00000100:00100000:1.0:1789760775.617516:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754684424720 time 754684164304 state 0x1 ref 1 00000100:00100000:1.0:1789760775.617576:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754684424720 time 754722285642 state 0x0 ref 1 00000100:00100000:1.0:1789760775.617589:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508433536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760775.617644:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23949312, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:1.0:1789760775.617653:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.617668:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.617673:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.617680:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.617694:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.650112:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973167 is committed 00002000:00010000:1.0:1789760775.650221:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23949312 -> 23953408 00010000:00010000:1.0:1789760775.650237:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.650251:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.650260:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.650268:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760775.650273:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.650281:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.650332:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.650341:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23949312->23953407] (req 23949312->23953407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.650436:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508433536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32843us (33206us total) trans 4294973167 rc 0/0 00000100:00100000:1.0:1789760775.650447:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5879 00000100:00100000:1.0:1789760775.650456:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754684424720 time 754722285642 state 0x2 ref 0 00000100:00100000:0.0:1789760775.655669:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760775.655804:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508433664 00000100:00100000:3.0:1789760775.655813:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508433664 00000100:00100000:3.0:1789760775.655878:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754684424720 time 754722285642 state 0x0 ref 1 00000100:00100000:3.0:1789760775.655893:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754722546058 time 754722285642 state 0x1 ref 1 00000100:00100000:3.0:1789760775.655951:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754722546058 time 754760660881 state 0x0 ref 1 00000100:00100000:3.0:1789760775.655964:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508433664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760775.656010:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23953408, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:3.0:1789760775.656019:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.656034:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.656038:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.656045:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.656070:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.694793:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23953408 -> 23957504 00010000:00010000:3.0:1789760775.694814:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.694839:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.694854:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.694862:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760775.694867:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.694875:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.694896:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.694904:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe566f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23953408->23957503] (req 23953408->23957503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.694996:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508433664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39028us (39331us total) trans 4294973168 rc 0/0 00000100:00100000:3.0:1789760775.695006:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5880 00000100:00100000:3.0:1789760775.695016:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754722546058 time 754760660881 state 0x2 ref 0 00000001:00080000:1.0:1789760775.696088:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973168 is committed 00000100:00100000:2.0:1789760775.698587:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760775.698710:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508433792 00000100:00100000:1.0:1789760775.698720:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508433792 00000100:00100000:1.0:1789760775.698784:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754722546058 time 754760660881 state 0x0 ref 1 00000100:00100000:1.0:1789760775.698800:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754760921297 time 754760660881 state 0x1 ref 1 00000100:00100000:1.0:1789760775.698870:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754760921297 time 754803579779 state 0x0 ref 1 00000100:00100000:1.0:1789760775.698883:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7343b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508433792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760775.698932:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23957504, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:1.0:1789760775.698940:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.698955:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.698960:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.698966:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.698979:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.728061:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973169 is committed 00002000:00010000:0.0:1789760775.728208:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23957504 -> 23961600 00010000:00010000:0.0:1789760775.728220:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.728230:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.728237:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.728241:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760775.728246:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.728255:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.728280:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.728307:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe566fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23957504->23961599] (req 23957504->23961599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.728396:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7343b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508433792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29510us (29812us total) trans 4294973169 rc 0/0 00000100:00100000:0.0:1789760775.728407:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5881 00000100:00100000:0.0:1789760775.728416:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754760921297 time 754803579779 state 0x2 ref 0 00000100:00100000:2.0:1789760775.731701:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760775.731841:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508433920 00000100:00100000:1.0:1789760775.731851:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508433920 00000100:00100000:1.0:1789760775.731902:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754760921297 time 754803579779 state 0x0 ref 1 00000100:00100000:1.0:1789760775.731915:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754803840195 time 754803579779 state 0x1 ref 1 00000100:00100000:1.0:1789760775.731938:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754803840195 time 754836648793 state 0x0 ref 1 00000100:00100000:1.0:1789760775.731950:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508433920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760775.731996:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23961600, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:1.0:1789760775.732004:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56705 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.732018:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.732022:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.732028:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56705 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.732041:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56705 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760775.766145:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23961600 -> 23965696 00010000:00010000:1.0:1789760775.766176:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56705 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.766191:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56705 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.766201:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56705 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.766207:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760775.766213:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56705 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.766222:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56705 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.766242:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56705 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.766250:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56705 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23961600->23965695] (req 23961600->23965695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.766380:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508433920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34425us (34680us total) trans 4294973170 rc 0/0 00000100:00100000:1.0:1789760775.766391:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5882 00000100:00100000:1.0:1789760775.766402:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754803840195 time 754836648793 state 0x2 ref 0 00000001:00080000:2.0:1789760775.768522:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973170 is committed 00000100:00100000:1.0:1789760775.769330:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760775.769406:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508434048 00000100:00100000:3.0:1789760775.769415:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508434048 00000100:00100000:3.0:1789760775.769467:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754803840195 time 754836648793 state 0x0 ref 1 00000100:00100000:3.0:1789760775.769482:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754836909209 time 754836648793 state 0x1 ref 1 00000100:00100000:3.0:1789760775.769504:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754836909209 time 754874214991 state 0x0 ref 1 00000100:00100000:3.0:1789760775.769516:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508434048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760775.769562:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23965696, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:3.0:1789760775.769571:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5670c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.769586:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.769590:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.769596:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5670c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.769609:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5670c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.794339:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23965696 -> 23969792 00000001:00080000:0.0:1789760775.794357:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973171 is committed 00010000:00010000:3.0:1789760775.794360:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5670c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.794373:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5670c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.794382:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5670c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.794390:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760775.794396:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5670c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.794405:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5670c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.794424:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5670c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.794429:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe5670c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23965696->23969791] (req 23965696->23969791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.794517:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508434048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24998us (25187us total) trans 4294973171 rc 0/0 00000100:00100000:3.0:1789760775.794527:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5883 00000100:00100000:3.0:1789760775.794536:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754836909209 time 754874214991 state 0x2 ref 0 00000100:00100000:0.0:1789760775.799081:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760775.799330:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508434176 00000100:00100000:1.0:1789760775.799340:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508434176 00000100:00100000:1.0:1789760775.799395:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754836909209 time 754874214991 state 0x0 ref 1 00000100:00100000:1.0:1789760775.799416:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754874475407 time 754874214991 state 0x1 ref 1 00000100:00100000:1.0:1789760775.799440:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754874475407 time 754904151251 state 0x0 ref 1 00000100:00100000:1.0:1789760775.799452:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508434176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760775.799504:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23969792, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:1.0:1789760775.799512:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56713 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.799526:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.799531:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.799538:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56713 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.799552:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56713 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760775.829989:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23969792 -> 23973888 00010000:00010000:3.0:1789760775.830009:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56713 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.830022:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56713 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.830031:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56713 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.830044:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760775.830049:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56713 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.830063:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56713 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.830083:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56713 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.830091:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56713 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23969792->23973887] (req 23969792->23973887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.830110:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973172 is committed 00000100:00100000:3.0:1789760775.830245:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508434176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30726us (31105us total) trans 4294973172 rc 0/0 00000100:00100000:3.0:1789760775.830256:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5884 00000100:00100000:3.0:1789760775.830268:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754874475407 time 754904151251 state 0x2 ref 0 00000100:00100000:2.0:1789760775.836711:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760775.837363:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508434304 00000100:00100000:1.0:1789760775.837374:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508434304 00000100:00100000:1.0:1789760775.837429:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754874475407 time 754904151251 state 0x0 ref 1 00000100:00100000:1.0:1789760775.837443:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754904411667 time 754904151251 state 0x1 ref 1 00000100:00100000:1.0:1789760775.837465:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754904411667 time 754942175341 state 0x0 ref 1 00000100:00100000:1.0:1789760775.837476:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508434304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760775.837524:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23973888, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:1.0:1789760775.837533:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5671a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.837549:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:1.0:1789760775.837554:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.837560:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5671a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.837574:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5671a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.872712:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973173 is committed 00002000:00010000:3.0:1789760775.872841:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23973888 -> 23977984 00010000:00010000:3.0:1789760775.872928:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5671a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.872942:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5671a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.872958:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5671a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.872967:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760775.872973:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5671a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.872982:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5671a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.873004:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5671a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.873013:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5671a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23973888->23977983] (req 23973888->23977983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.873161:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508434304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35680us (36454us total) trans 4294973173 rc 0/0 00000100:00100000:3.0:1789760775.873183:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5885 00000100:00100000:3.0:1789760775.873195:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754904411667 time 754942175341 state 0x2 ref 0 00000100:00100000:1.0:1789760775.879410:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760775.879570:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508434432 00000100:00100000:3.0:1789760775.879580:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508434432 00000100:00100000:3.0:1789760775.879632:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754904411667 time 754942175341 state 0x0 ref 1 00000100:00100000:3.0:1789760775.879647:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754942435757 time 754942175341 state 0x1 ref 1 00000100:00100000:3.0:1789760775.879720:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754942435757 time 754984429351 state 0x0 ref 1 00000100:00100000:3.0:1789760775.879743:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508434432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760775.879792:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23977984, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:3.0:1789760775.879801:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56721 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.879826:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=754s v=(0 0) 00010000:00010000:3.0:1789760775.879831:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.879838:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56721 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.879852:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56721 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.903806:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973174 is committed 00002000:00010000:0.0:1789760775.903893:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23977984 -> 23982080 00010000:00010000:0.0:1789760775.903908:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56721 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.903921:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56721 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.903929:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56721 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.903935:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760775.903941:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56721 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.903950:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56721 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.903971:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56721 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.903979:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56721 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23977984->23982079] (req 23977984->23982079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.904102:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508434432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24365us (24698us total) trans 4294973174 rc 0/0 00000100:00100000:0.0:1789760775.904111:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5886 00000100:00100000:0.0:1789760775.904121:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754942435757 time 754984429351 state 0x2 ref 0 00000100:00100000:1.0:1789760775.909122:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760775.909569:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508434560 00000100:00100000:3.0:1789760775.909579:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508434560 00000100:00100000:3.0:1789760775.909634:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754942435757 time 754984429351 state 0x0 ref 1 00000100:00100000:3.0:1789760775.909649:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754984689767 time 754984429351 state 0x1 ref 1 00000100:00100000:3.0:1789760775.909921:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754984689767 time 755014629774 state 0x0 ref 1 00000100:00100000:3.0:1789760775.909937:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508434560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760775.909983:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23982080, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:3.0:1789760775.909993:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56728 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.910008:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760775.910012:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.910019:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56728 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.910033:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56728 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.934762:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760775.950183:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973175 is committed 00002000:00010000:0.0:1789760775.950267:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23982080 -> 23986176 00010000:00010000:0.0:1789760775.950381:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56728 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.950398:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56728 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.950406:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56728 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.950419:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760775.950424:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56728 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.950435:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56728 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.950455:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56728 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.950599:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56728 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23982080->23986175] (req 23982080->23986175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.950695:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508434560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40757us (41578us total) trans 4294973175 rc 0/0 00000100:00100000:0.0:1789760775.950707:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5887 00000100:00100000:0.0:1789760775.950717:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754984689767 time 755014629774 state 0x2 ref 0 00000100:00100000:2.0:1789760775.954491:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760775.954597:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508434688 00000100:00100000:3.0:1789760775.954608:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508434688 00000100:00100000:3.0:1789760775.954664:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 754984689767 time 755014629774 state 0x0 ref 1 00000100:00100000:3.0:1789760775.954680:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755014890190 time 755014629774 state 0x1 ref 1 00000100:00100000:3.0:1789760775.954710:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755014890190 time 755059421127 state 0x0 ref 1 00000100:00100000:3.0:1789760775.954723:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508434688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760775.954770:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23986176, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:3.0:1789760775.954779:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5672f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.954793:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760775.954798:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.954804:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5672f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.954831:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5672f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.986359:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973176 is committed 00002000:00010000:3.0:1789760775.986900:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23986176 -> 23990272 00010000:00010000:3.0:1789760775.986915:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5672f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.986928:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5672f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.986938:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5672f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.986945:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760775.986950:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5672f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.986958:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5672f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.986977:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5672f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.986985:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5672f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23986176->23990271] (req 23986176->23990271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.987193:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508434688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32467us (32707us total) trans 4294973176 rc 0/0 00000100:00100000:3.0:1789760775.987203:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5888 00000100:00100000:3.0:1789760775.987212:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755014890190 time 755059421127 state 0x2 ref 0 00000100:00100000:1.0:1789760775.994136:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760775.994622:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508434816 00000100:00100000:0.0:1789760775.994632:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508434816 00000100:00100000:0.0:1789760775.994686:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755014890190 time 755059421127 state 0x0 ref 1 00000100:00100000:0.0:1789760775.994702:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755059681543 time 755059421127 state 0x1 ref 1 00000100:00100000:0.0:1789760775.994735:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755059681543 time 755099445064 state 0x0 ref 1 00000100:00100000:0.0:1789760775.994870:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508434816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760775.994917:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23990272, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:0.0:1789760775.994925:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56736 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.994938:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760775.994942:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.994947:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56736 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.994959:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56736 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760776.028933:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23990272 -> 23994368 00010000:00010000:3.0:1789760776.028953:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56736 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.028966:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56736 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.028976:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56736 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.028983:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760776.028989:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56736 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.028998:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56736 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.029018:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56736 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.029030:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56736 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23990272->23994367] (req 23990272->23994367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.029162:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508434816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34293us (35031us total) trans 4294973177 rc 0/0 00000100:00100000:3.0:1789760776.029237:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5889 00000100:00100000:3.0:1789760776.029247:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755059681543 time 755099445064 state 0x2 ref 0 00000001:00080000:0.0:1789760776.029380:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973177 is committed 00000100:00100000:1.0:1789760776.032562:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760776.032999:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508434944 00000100:00100000:0.0:1789760776.033072:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508434944 00000100:00100000:0.0:1789760776.033128:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755059681543 time 755099445064 state 0x0 ref 1 00000100:00100000:0.0:1789760776.033144:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755099705480 time 755099445064 state 0x1 ref 1 00000100:00100000:0.0:1789760776.033276:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755099705480 time 755137985318 state 0x0 ref 1 00000100:00100000:0.0:1789760776.033328:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508434944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760776.033376:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23994368, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:0.0:1789760776.033387:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5673d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.033408:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760776.033412:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.033418:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5673d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.033433:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5673d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.074979:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973178 is committed 00002000:00010000:1.0:1789760776.075081:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23994368 -> 23998464 00010000:00010000:1.0:1789760776.075096:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5673d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.075110:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5673d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.075119:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5673d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.075127:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760776.075132:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5673d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.075140:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5673d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.075162:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5673d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.075179:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5673d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23994368->23998463] (req 23994368->23998463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.075506:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508434944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42176us (42945us total) trans 4294973178 rc 0/0 00000100:00100000:1.0:1789760776.075517:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5890 00000100:00100000:1.0:1789760776.075527:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755099705480 time 755137985318 state 0x2 ref 0 00000100:00100000:2.0:1789760776.080533:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760776.080615:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508435072 00000100:00100000:3.0:1789760776.080624:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508435072 00000100:00100000:3.0:1789760776.080678:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755099705480 time 755137985318 state 0x0 ref 1 00000100:00100000:3.0:1789760776.080693:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755138245734 time 755137985318 state 0x1 ref 1 00000100:00100000:3.0:1789760776.080728:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755138245734 time 755185437803 state 0x0 ref 1 00000100:00100000:3.0:1789760776.080740:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508435072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760776.080790:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 23998464, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:3.0:1789760776.080800:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56744 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.080825:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.080830:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.080836:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56744 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.080856:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56744 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.115103:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973179 is committed 00002000:00010000:3.0:1789760776.115340:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 23998464 -> 24002560 00010000:00010000:3.0:1789760776.115355:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56744 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.115368:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56744 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.115378:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56744 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.115385:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760776.115391:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56744 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.115401:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56744 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.115422:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56744 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.115430:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56744 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [23998464->24002559] (req 23998464->24002559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.115565:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508435072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34821us (35176us total) trans 4294973179 rc 0/0 00000100:00100000:3.0:1789760776.115576:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5891 00000100:00100000:3.0:1789760776.115586:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755138245734 time 755185437803 state 0x2 ref 0 00000100:00100000:3.0:1789760776.123541:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760776.123682:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508435200 00000100:00100000:0.0:1789760776.123692:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508435200 00000100:00100000:0.0:1789760776.123745:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755138245734 time 755185437803 state 0x0 ref 1 00000100:00100000:0.0:1789760776.123760:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755185698219 time 755185437803 state 0x1 ref 1 00000100:00100000:0.0:1789760776.123791:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755185698219 time 755228501377 state 0x0 ref 1 00000100:00100000:0.0:1789760776.123802:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508435200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760776.123859:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24002560, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:0.0:1789760776.123868:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5674b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.123883:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760776.123888:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.123894:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5674b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.123909:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5674b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760776.147791:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24002560 -> 24006656 00010000:00010000:0.0:1789760776.147810:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5674b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.147835:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5674b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.147844:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5674b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.147851:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760776.147856:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5674b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.147865:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5674b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.147886:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5674b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.147894:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5674b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24002560->24006655] (req 24002560->24006655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.147988:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508435200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24182us (24451us total) trans 4294973180 rc 0/0 00000100:00100000:0.0:1789760776.147998:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5892 00000100:00100000:0.0:1789760776.148008:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755185698219 time 755228501377 state 0x2 ref 0 00000001:00080000:0.0:1789760776.148047:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973180 is committed 00000100:00100000:1.0:1789760776.151832:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760776.151911:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508435328 00000100:00100000:0.0:1789760776.151920:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508435328 00000100:00100000:0.0:1789760776.151973:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755185698219 time 755228501377 state 0x0 ref 1 00000100:00100000:0.0:1789760776.151988:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755228761793 time 755228501377 state 0x1 ref 1 00000100:00100000:0.0:1789760776.152021:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755228761793 time 755256731249 state 0x0 ref 1 00000100:00100000:0.0:1789760776.152034:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508435328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760776.152083:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24006656, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:0.0:1789760776.152092:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56752 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.152108:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760776.152112:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.152119:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56752 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.152133:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56752 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.178058:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973181 is committed 00002000:00010000:0.0:1789760776.178157:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24006656 -> 24010752 00010000:00010000:0.0:1789760776.178182:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56752 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.178195:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56752 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.178204:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56752 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.178211:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760776.178216:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56752 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.178225:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56752 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.178243:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56752 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.178250:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56752 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24006656->24010751] (req 24006656->24010751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.178384:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508435328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26347us (26567us total) trans 4294973181 rc 0/0 00000100:00100000:0.0:1789760776.178397:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5893 00000100:00100000:0.0:1789760776.178407:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755228761793 time 755256731249 state 0x2 ref 0 00000100:00100000:2.0:1789760776.181447:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760776.181527:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508435456 00000100:00100000:3.0:1789760776.181537:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508435456 00000100:00100000:3.0:1789760776.181590:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755228761793 time 755256731249 state 0x0 ref 1 00000100:00100000:3.0:1789760776.181606:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755256991665 time 755256731249 state 0x1 ref 1 00000100:00100000:3.0:1789760776.181636:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755256991665 time 755286346672 state 0x0 ref 1 00000100:00100000:3.0:1789760776.181648:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508435456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760776.181695:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24010752, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:3.0:1789760776.181705:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56759 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.181720:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.181724:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.181742:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56759 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.181761:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56759 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.213043:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973182 is committed 00002000:00010000:3.0:1789760776.213139:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24010752 -> 24014848 00010000:00010000:3.0:1789760776.213154:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56759 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.213177:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56759 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.213186:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56759 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.213193:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760776.213197:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56759 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.213206:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56759 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.213225:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56759 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.213233:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56759 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24010752->24014847] (req 24010752->24014847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.213499:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508435456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31847us (32056us total) trans 4294973182 rc 0/0 00000100:00100000:3.0:1789760776.213509:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5894 00000100:00100000:3.0:1789760776.213519:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755256991665 time 755286346672 state 0x2 ref 0 00000100:00100000:2.0:1789760776.217219:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760776.219513:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508435584 00000100:00100000:0.0:1789760776.219527:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508435584 00000100:00100000:0.0:1789760776.219591:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755256991665 time 755286346672 state 0x0 ref 1 00000100:00100000:0.0:1789760776.219608:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755286607088 time 755286346672 state 0x1 ref 1 00000100:00100000:0.0:1789760776.219750:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755286607088 time 755324458940 state 0x0 ref 1 00000100:00100000:0.0:1789760776.219764:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508435584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760776.219824:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24014848, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:0.0:1789760776.219833:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56760 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.219849:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760776.219853:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.219859:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56760 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.219873:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56760 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.257069:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760776.257103:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760776.257131:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760776.257152:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760776.257173:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760776.257199:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508435840 00000100:00100000:0.0:1789760776.257209:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508435840 00000100:00100000:0.0:1789760776.257261:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508436096 00000100:00100000:0.0:1789760776.257264:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508436096 00000100:00100000:0.0:1789760776.257281:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508435968 00000100:00100000:0.0:1789760776.257305:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508435968 00000100:00100000:0.0:1789760776.257319:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508436224 00000100:00100000:0.0:1789760776.257321:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508436224 00000100:00100000:0.0:1789760776.257335:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 959 00000100:00100000:0.0:1789760776.257343:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e67b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2406:x1876699508435840:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760776.257415:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e67b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2406:x1876699508435840:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 72us (355us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760776.257422:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 959 00000100:00100000:0.0:1789760776.257438:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 960 00000100:00100000:0.0:1789760776.257445:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2406:x1876699508436096:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760776.257471:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2406:x1876699508436096:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 27us (368us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760776.257476:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 960 00000100:00100000:0.0:1789760776.257488:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 961 00000100:00100000:0.0:1789760776.257493:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508435968:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760776.257517:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508435968:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 24us (365us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760776.257519:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 962 00000100:00100000:0.0:1789760776.257524:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 961 00000100:00100000:1.0:1789760776.257528:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2406:x1876699508436224:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760776.257558:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2406:x1876699508436224:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 31us (384us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760776.257564:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 962 00000100:00080000:1.0:1789760776.257580:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760776.257584:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760776.257708:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508435712 00000100:00100000:2.0:1789760776.257712:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508435712 00000100:00100000:2.0:1789760776.258382:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 485 00000100:00100000:2.0:1789760776.258393:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cadc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2406:x1876699508435712:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760776.258449:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cadc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2406:x1876699508435712:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 59us (1318us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760776.258457:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 485 00000001:00080000:3.0:1789760776.268483:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973183 is committed 00002000:00010000:2.0:1789760776.268551:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24014848 -> 24018944 00010000:00010000:2.0:1789760776.268566:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56760 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.268579:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56760 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.268589:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56760 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.268596:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760776.268600:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56760 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.268609:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56760 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.268629:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56760 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.268634:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56760 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24014848->24018943] (req 24014848->24018943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.268826:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508435584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49049us (51658us total) trans 4294973183 rc 0/0 00000100:00100000:2.0:1789760776.268838:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5895 00000100:00100000:2.0:1789760776.268849:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755286607088 time 755324458940 state 0x2 ref 0 00000100:00100000:0.0:1789760776.272590:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760776.272663:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508436352 00000100:00100000:1.0:1789760776.272675:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508436352 00000100:00100000:1.0:1789760776.272729:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755286607088 time 755324458940 state 0x0 ref 1 00000100:00100000:1.0:1789760776.272837:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755324719356 time 755324458940 state 0x1 ref 1 00000100:00100000:1.0:1789760776.272878:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755324719356 time 755377588223 state 0x0 ref 1 00000100:00100000:1.0:1789760776.272892:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508436352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760776.272936:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24018944, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:1.0:1789760776.272945:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56767 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.272960:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:1.0:1789760776.272965:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.272971:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56767 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.272986:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56767 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.308466:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973184 is committed 00002000:00010000:1.0:1789760776.308551:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24018944 -> 24023040 00010000:00010000:1.0:1789760776.308568:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56767 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.308580:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56767 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.308589:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56767 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.308596:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760776.308602:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56767 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.308611:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56767 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.308630:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56767 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.308637:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56767 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24018944->24023039] (req 24018944->24023039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.308726:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508436352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35833us (36145us total) trans 4294973184 rc 0/0 00000100:00100000:1.0:1789760776.309560:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5896 00000100:00100000:1.0:1789760776.309578:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755324719356 time 755377588223 state 0x2 ref 0 00000100:00100000:1.0:1789760776.312276:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760776.312558:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508436480 00000100:00100000:3.0:1789760776.312569:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508436480 00000100:00100000:2.0:1789760776.312692:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755324719356 time 755377588223 state 0x0 ref 1 00000100:00100000:2.0:1789760776.312705:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755377848639 time 755377588223 state 0x1 ref 1 00000100:00100000:2.0:1789760776.312741:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755377848639 time 755417451132 state 0x0 ref 1 00000100:00100000:2.0:1789760776.312753:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508436480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760776.312800:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24023040, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:2.0:1789760776.312809:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5676e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.312835:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760776.312840:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.312846:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5676e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.312860:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5676e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.339832:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973185 is committed 00002000:00010000:2.0:1789760776.339854:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24023040 -> 24027136 00010000:00010000:2.0:1789760776.339869:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5676e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.339882:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5676e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.339895:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5676e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.339902:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760776.339907:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5676e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.339916:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5676e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.339934:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5676e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.339941:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe5676e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24023040->24027135] (req 24023040->24027135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.340038:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508436480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27280us (27766us total) trans 4294973185 rc 0/0 00000100:00100000:2.0:1789760776.340047:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5897 00000100:00100000:2.0:1789760776.340057:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755377848639 time 755417451132 state 0x2 ref 0 00000100:00100000:0.0:1789760776.347864:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760776.347983:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508436608 00000100:00100000:2.0:1789760776.347993:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508436608 00000100:00100000:2.0:1789760776.348045:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755377848639 time 755417451132 state 0x0 ref 1 00000100:00100000:2.0:1789760776.348060:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755417711548 time 755417451132 state 0x1 ref 1 00000100:00100000:2.0:1789760776.348134:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755417711548 time 755452844449 state 0x0 ref 1 00000100:00100000:2.0:1789760776.348146:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508436608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760776.348196:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24027136, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:2.0:1789760776.348205:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56775 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.348219:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760776.348223:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.348229:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56775 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.348242:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56775 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.381821:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973186 is committed 00002000:00010000:0.0:1789760776.381919:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24027136 -> 24031232 00010000:00010000:0.0:1789760776.381934:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56775 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.381945:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56775 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.381955:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56775 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.381962:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760776.381967:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56775 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.381977:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56775 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.381998:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56775 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.382007:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56775 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24027136->24031231] (req 24027136->24031231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.382563:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508436608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34412us (34703us total) trans 4294973186 rc 0/0 00000100:00100000:0.0:1789760776.382577:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5898 00000100:00100000:0.0:1789760776.382588:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755417711548 time 755452844449 state 0x2 ref 0 00000100:00100000:1.0:1789760776.391636:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760776.391713:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508436736 00000100:00100000:0.0:1789760776.391723:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508436736 00000100:00100000:0.0:1789760776.391774:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755417711548 time 755452844449 state 0x0 ref 1 00000100:00100000:0.0:1789760776.391788:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755453104865 time 755452844449 state 0x1 ref 1 00000100:00100000:0.0:1789760776.391860:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755453104865 time 755496569512 state 0x0 ref 1 00000100:00100000:0.0:1789760776.391873:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d43850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508436736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760776.391919:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24031232, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:0.0:1789760776.391927:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5677c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.391942:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760776.391946:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.391953:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5677c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.391967:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5677c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.429727:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973187 is committed 00002000:00010000:3.0:1789760776.429826:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24031232 -> 24035328 00010000:00010000:3.0:1789760776.429843:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5677c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.429855:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5677c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.429863:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5677c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.429870:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760776.429874:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5677c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.429883:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5677c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.429910:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5677c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.429918:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5677c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24031232->24035327] (req 24031232->24035327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.430052:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d43850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508436736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38175us (38419us total) trans 4294973187 rc 0/0 00000100:00100000:3.0:1789760776.430063:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5899 00000100:00100000:3.0:1789760776.430073:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755453104865 time 755496569512 state 0x2 ref 0 00000100:00100000:1.0:1789760776.435864:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760776.436189:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508436864 00000100:00100000:2.0:1789760776.436199:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508436864 00000100:00100000:2.0:1789760776.436318:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755453104865 time 755496569512 state 0x0 ref 1 00000100:00100000:2.0:1789760776.436334:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755496829928 time 755496569512 state 0x1 ref 1 00000100:00100000:2.0:1789760776.436356:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755496829928 time 755541067666 state 0x0 ref 1 00000100:00100000:2.0:1789760776.436368:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508436864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760776.436413:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24035328, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:2.0:1789760776.436423:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56783 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.436438:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760776.436442:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.436449:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56783 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.436467:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56783 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.469494:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973188 is committed 00002000:00010000:2.0:1789760776.469560:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24035328 -> 24039424 00010000:00010000:2.0:1789760776.469574:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56783 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.469586:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56783 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.469595:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56783 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.469608:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760776.469613:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56783 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.469621:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56783 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.469640:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56783 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.469649:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56783 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24035328->24039423] (req 24035328->24039423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.469730:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508436864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33360us (33870us total) trans 4294973188 rc 0/0 00000100:00100000:2.0:1789760776.469740:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5900 00000100:00100000:2.0:1789760776.469749:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755496829928 time 755541067666 state 0x2 ref 0 00000100:00100000:0.0:1789760776.474478:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760776.474618:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508436992 00000100:00100000:2.0:1789760776.474629:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508436992 00000100:00100000:2.0:1789760776.474685:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755496829928 time 755541067666 state 0x0 ref 1 00000100:00100000:2.0:1789760776.474702:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755541328082 time 755541067666 state 0x1 ref 1 00000100:00100000:2.0:1789760776.474733:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755541328082 time 755579443454 state 0x0 ref 1 00000100:00100000:2.0:1789760776.474748:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508436992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760776.474798:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24039424, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0xb807 00010000:00010000:2.0:1789760776.474808:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5678a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.474833:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760776.474837:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.474844:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5678a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.475557:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5678a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.520394:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973189 is committed 00002000:00010000:2.0:1789760776.520721:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24039424 -> 24043520 00002000:00010000:2.0:1789760776.520726:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760776 -> 1789760777 00002000:00010000:2.0:1789760776.520729:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760776 -> 1789760777 00010000:00010000:2.0:1789760776.520742:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5678a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.520755:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5678a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.520764:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5678a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.520771:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760776.520776:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5678a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.520786:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5678a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.520805:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5678a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.520823:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5678a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24039424->24043519] (req 24039424->24043519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.520916:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508436992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46166us (46441us total) trans 4294973189 rc 0/0 00000100:00100000:2.0:1789760776.520926:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5901 00000100:00100000:2.0:1789760776.520935:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755541328082 time 755579443454 state 0x2 ref 0 00000100:00100000:2.0:1789760776.529936:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760776.530343:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508437120 00000100:00100000:3.0:1789760776.530353:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508437120 00000100:00100000:3.0:1789760776.530424:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755541328082 time 755579443454 state 0x0 ref 1 00000100:00100000:3.0:1789760776.530594:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755579703870 time 755579443454 state 0x1 ref 1 00000100:00100000:3.0:1789760776.530633:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755579703870 time 755635343268 state 0x0 ref 1 00000100:00100000:3.0:1789760776.530646:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508437120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760776.530696:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24043520, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:3.0:1789760776.530705:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56791 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.530721:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.530737:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.530745:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56791 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.530760:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56791 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.575894:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973190 is committed 00002000:00010000:3.0:1789760776.575996:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24043520 -> 24047616 00010000:00010000:3.0:1789760776.576011:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56791 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.576023:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56791 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.576032:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56791 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.576038:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760776.576043:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56791 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.576051:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56791 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.576071:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56791 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.576079:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56791 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24043520->24047615] (req 24043520->24047615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.576249:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508437120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45600us (46318us total) trans 4294973190 rc 0/0 00000100:00100000:3.0:1789760776.576259:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5902 00000100:00100000:3.0:1789760776.576269:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755579703870 time 755635343268 state 0x2 ref 0 00000100:00100000:2.0:1789760776.580109:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760776.580199:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508437248 00000100:00100000:3.0:1789760776.580209:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508437248 00000100:00100000:3.0:1789760776.580263:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755579703870 time 755635343268 state 0x0 ref 1 00000100:00100000:3.0:1789760776.580277:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755635603684 time 755635343268 state 0x1 ref 1 00000100:00100000:0.0:1789760776.580662:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755635603684 time 755685370334 state 0x0 ref 1 00000100:00100000:0.0:1789760776.580677:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508437248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760776.580730:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24047616, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:0.0:1789760776.580739:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56798 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.580756:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760776.580760:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.580767:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56798 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.580781:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56798 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.607141:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973191 is committed 00002000:00010000:2.0:1789760776.607516:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24047616 -> 24051712 00010000:00010000:2.0:1789760776.607534:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56798 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.607547:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56798 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.607557:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56798 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.607565:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760776.607570:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56798 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.607579:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56798 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.607598:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56798 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.607606:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56798 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24047616->24051711] (req 24047616->24051711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.607740:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508437248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27060us (27636us total) trans 4294973191 rc 0/0 00000100:00100000:2.0:1789760776.607751:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5903 00000100:00100000:2.0:1789760776.607761:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755635603684 time 755685370334 state 0x2 ref 0 00000100:00100000:0.0:1789760776.610925:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760776.611049:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508437376 00000100:00100000:2.0:1789760776.611059:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508437376 00000100:00100000:2.0:1789760776.611127:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755635603684 time 755685370334 state 0x0 ref 1 00000100:00100000:2.0:1789760776.611143:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755685630750 time 755685370334 state 0x1 ref 1 00000100:00100000:2.0:1789760776.611170:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755685630750 time 755715881569 state 0x0 ref 1 00000100:00100000:2.0:1789760776.611185:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508437376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760776.611236:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24051712, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:2.0:1789760776.611246:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5679f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.611261:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760776.611266:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.611273:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5679f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.611316:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5679f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.640399:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973192 is committed 00002000:00010000:2.0:1789760776.640581:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24051712 -> 24055808 00010000:00010000:2.0:1789760776.640596:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5679f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.640609:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5679f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.640618:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5679f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.640626:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760776.640630:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5679f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.640639:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5679f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.640661:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5679f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.640668:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe5679f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24051712->24055807] (req 24051712->24055807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.640772:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508437376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29584us (29851us total) trans 4294973192 rc 0/0 00000100:00100000:2.0:1789760776.640782:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5904 00000100:00100000:2.0:1789760776.640793:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755685630750 time 755715881569 state 0x2 ref 0 00000100:00100000:1.0:1789760776.647045:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760776.647172:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508437504 00000100:00100000:3.0:1789760776.647182:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508437504 00000100:00100000:3.0:1789760776.647317:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755685630750 time 755715881569 state 0x0 ref 1 00000100:00100000:3.0:1789760776.647335:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755716141985 time 755715881569 state 0x1 ref 1 00000100:00100000:3.0:1789760776.647367:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755716141985 time 755752077141 state 0x0 ref 1 00000100:00100000:3.0:1789760776.647382:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508437504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760776.648556:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24055808, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:3.0:1789760776.648569:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.648585:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.648590:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.648598:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.648612:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760776.687143:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24055808 -> 24059904 00010000:00010000:3.0:1789760776.687164:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.687178:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.687188:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.687196:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760776.687201:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.687210:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.687231:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.687239:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24055808->24059903] (req 24055808->24059903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.687268:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973193 is committed 00000100:00100000:3.0:1789760776.687369:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508437504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39985us (40330us total) trans 4294973193 rc 0/0 00000100:00100000:3.0:1789760776.687379:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5905 00000100:00100000:3.0:1789760776.687388:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755716141985 time 755752077141 state 0x2 ref 0 00000100:00100000:1.0:1789760776.691451:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760776.691626:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508437632 00000100:00100000:2.0:1789760776.691635:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508437632 00000100:00100000:2.0:1789760776.691692:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755716141985 time 755752077141 state 0x0 ref 1 00000100:00100000:2.0:1789760776.691713:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755752337557 time 755752077141 state 0x1 ref 1 00000100:00100000:2.0:1789760776.691778:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755752337557 time 755796487456 state 0x0 ref 1 00000100:00100000:2.0:1789760776.691791:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508437632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760776.691853:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24059904, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:2.0:1789760776.691862:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe567ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.691878:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760776.691883:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.691890:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe567ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.691905:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe567ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760776.731989:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24059904 -> 24064000 00010000:00010000:3.0:1789760776.732011:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe567ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.732025:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe567ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.732035:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe567ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.732042:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760776.732047:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe567ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.732055:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe567ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.732075:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe567ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.732088:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe567ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24059904->24063999] (req 24059904->24063999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.732347:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508437632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40553us (40900us total) trans 4294973194 rc 0/0 00000100:00100000:3.0:1789760776.732357:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5906 00000100:00100000:3.0:1789760776.732366:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755752337557 time 755796487456 state 0x2 ref 0 00000001:00080000:1.0:1789760776.732506:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973194 is committed 00000100:00100000:0.0:1789760776.735574:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760776.735662:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508437760 00000100:00100000:2.0:1789760776.735671:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508437760 00000100:00100000:2.0:1789760776.735793:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755752337557 time 755796487456 state 0x0 ref 1 00000100:00100000:2.0:1789760776.735818:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755796747872 time 755796487456 state 0x1 ref 1 00000100:00100000:2.0:1789760776.735849:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755796747872 time 755840560162 state 0x0 ref 1 00000100:00100000:2.0:1789760776.735861:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508437760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760776.735908:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24064000, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:2.0:1789760776.735916:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe567b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.735932:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:2.0:1789760776.735937:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.735944:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe567b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.735964:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe567b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.764117:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973195 is committed 00002000:00010000:2.0:1789760776.764120:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24064000 -> 24068096 00010000:00010000:2.0:1789760776.764136:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe567b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.764149:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe567b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.764158:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe567b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.764166:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760776.764172:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe567b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.764180:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe567b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.764201:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe567b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.764215:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe567b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24064000->24068095] (req 24064000->24068095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.764423:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508437760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28559us (28854us total) trans 4294973195 rc 0/0 00000100:00100000:2.0:1789760776.764434:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5907 00000100:00100000:2.0:1789760776.764442:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755796747872 time 755840560162 state 0x2 ref 0 00000100:00100000:1.0:1789760776.767660:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760776.767822:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508437888 00000100:00100000:3.0:1789760776.767832:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508437888 00000100:00100000:3.0:1789760776.767892:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755796747872 time 755840560162 state 0x0 ref 1 00000100:00100000:3.0:1789760776.767906:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755840820578 time 755840560162 state 0x1 ref 1 00000100:00100000:3.0:1789760776.767987:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755840820578 time 755872697768 state 0x0 ref 1 00000100:00100000:3.0:1789760776.768000:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508437888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760776.768048:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24068096, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:3.0:1789760776.768056:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.768072:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.768076:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.768082:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.768102:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.795146:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973196 is committed 00002000:00010000:3.0:1789760776.795329:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24068096 -> 24072192 00010000:00010000:3.0:1789760776.795344:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.795357:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.795365:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.795372:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760776.795377:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.795386:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.795406:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.795414:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24068096->24072191] (req 24068096->24072191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.795554:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508437888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27550us (27898us total) trans 4294973196 rc 0/0 00000100:00100000:3.0:1789760776.795564:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5908 00000100:00100000:3.0:1789760776.795574:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755840820578 time 755872697768 state 0x2 ref 0 00000100:00100000:1.0:1789760776.798671:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760776.798750:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508438016 00000100:00100000:3.0:1789760776.798761:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508438016 00000100:00100000:3.0:1789760776.798828:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755840820578 time 755872697768 state 0x0 ref 1 00000100:00100000:3.0:1789760776.798843:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755872958184 time 755872697768 state 0x1 ref 1 00000100:00100000:3.0:1789760776.798873:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755872958184 time 755903584051 state 0x0 ref 1 00000100:00100000:3.0:1789760776.798884:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508438016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760776.798932:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24072192, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:3.0:1789760776.798940:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.798955:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:3.0:1789760776.798959:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.798965:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.798978:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.842968:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973197 is committed 00002000:00010000:3.0:1789760776.844414:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24072192 -> 24076288 00010000:00010000:3.0:1789760776.844433:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.844447:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.844455:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.844462:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760776.844466:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.844474:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.844492:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.844499:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24072192->24076287] (req 24072192->24076287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.844606:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508438016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45719us (45940us total) trans 4294973197 rc 0/0 00000100:00100000:3.0:1789760776.844620:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5909 00000100:00100000:3.0:1789760776.844629:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755872958184 time 755903584051 state 0x2 ref 0 00000100:00100000:2.0:1789760776.852596:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760776.853690:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508438144 00000100:00100000:0.0:1789760776.853702:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508438144 00000100:00100000:0.0:1789760776.853825:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755872958184 time 755903584051 state 0x0 ref 1 00000100:00100000:0.0:1789760776.853844:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755903844467 time 755903584051 state 0x1 ref 1 00000100:00100000:0.0:1789760776.853931:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755903844467 time 755958640422 state 0x0 ref 1 00000100:00100000:0.0:1789760776.853947:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508438144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760776.853995:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24076288, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:0.0:1789760776.854004:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe567c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.854020:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=755s v=(0 0) 00010000:00010000:0.0:1789760776.854024:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.854029:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe567c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.854044:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe567c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.894154:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973198 is committed 00002000:00010000:1.0:1789760776.894524:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24076288 -> 24080384 00010000:00010000:1.0:1789760776.894541:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe567c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.894555:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe567c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.894565:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe567c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.894573:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760776.894578:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe567c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.894587:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe567c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.894606:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe567c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.894615:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe567c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24076288->24080383] (req 24076288->24080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.895130:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508438144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41176us (42541us total) trans 4294973198 rc 0/0 00000100:00100000:1.0:1789760776.895141:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5910 00000100:00100000:1.0:1789760776.895151:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755903844467 time 755958640422 state 0x2 ref 0 00000100:00100000:1.0:1789760776.901239:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760776.903047:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508438272 00000100:00100000:2.0:1789760776.903062:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508438272 00000100:00100000:2.0:1789760776.903129:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755903844467 time 755958640422 state 0x0 ref 1 00000100:00100000:2.0:1789760776.903145:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755958900838 time 755958640422 state 0x1 ref 1 00000100:00100000:2.0:1789760776.903190:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755958900838 time 756007900955 state 0x0 ref 1 00000100:00100000:2.0:1789760776.903204:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508438272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760776.903253:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24080384, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:2.0:1789760776.903263:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.903279:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760776.903283:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.903317:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.903344:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.932659:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973199 is committed 00002000:00010000:2.0:1789760776.932765:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24080384 -> 24084480 00010000:00010000:2.0:1789760776.932780:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.932793:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.932802:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.932818:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760776.932824:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.932833:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.932851:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.932858:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24080384->24084479] (req 24080384->24084479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.933060:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508438272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29854us (31827us total) trans 4294973199 rc 0/0 00000100:00100000:2.0:1789760776.933074:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5911 00000100:00100000:2.0:1789760776.933083:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755958900838 time 756007900955 state 0x2 ref 0 00000100:00100000:2.0:1789760776.937672:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760776.937757:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508438400 00000100:00100000:3.0:1789760776.937767:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508438400 00000100:00100000:3.0:1789760776.937830:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 755958900838 time 756007900955 state 0x0 ref 1 00000100:00100000:3.0:1789760776.937847:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756008161371 time 756007900955 state 0x1 ref 1 00000100:00100000:3.0:1789760776.937883:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756008161371 time 756042592816 state 0x0 ref 1 00000100:00100000:3.0:1789760776.937896:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508438400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760776.937946:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24084480, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:3.0:1789760776.937956:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.937972:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760776.937977:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.937983:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.937998:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.960056:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760776.972503:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973200 is committed 00002000:00010000:2.0:1789760776.972696:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24084480 -> 24088576 00010000:00010000:2.0:1789760776.972711:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.972795:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.972805:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.972821:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760776.972826:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.972834:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.972853:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.972861:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe567d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24084480->24088575] (req 24084480->24088575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.972960:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508438400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35062us (35293us total) trans 4294973200 rc 0/0 00000100:00100000:2.0:1789760776.972971:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5912 00000100:00100000:2.0:1789760776.972980:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756008161371 time 756042592816 state 0x2 ref 0 00000100:00100000:1.0:1789760776.976220:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760776.976327:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508438528 00000100:00100000:2.0:1789760776.976336:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508438528 00000100:00100000:2.0:1789760776.976389:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756008161371 time 756042592816 state 0x0 ref 1 00000100:00100000:2.0:1789760776.976405:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756042853232 time 756042592816 state 0x1 ref 1 00000100:00100000:2.0:1789760776.976424:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756042853232 time 756081135358 state 0x0 ref 1 00000100:00100000:2.0:1789760776.976436:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508438528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760776.976482:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24088576, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:2.0:1789760776.976491:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.976507:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760776.976511:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.976517:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.976530:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.998998:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973201 is committed 00002000:00010000:3.0:1789760776.999114:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24088576 -> 24092672 00010000:00010000:3.0:1789760776.999132:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.999146:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.999155:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.999226:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760776.999231:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.999241:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.999259:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.999267:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe567de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24088576->24092671] (req 24088576->24092671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.999468:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508438528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23029us (23251us total) trans 4294973201 rc 0/0 00000100:00100000:3.0:1789760776.999477:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5913 00000100:00100000:3.0:1789760776.999487:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756042853232 time 756081135358 state 0x2 ref 0 00000100:00100000:1.0:1789760777.002562:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760777.002653:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508438656 00000100:00100000:2.0:1789760777.002663:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508438656 00000100:00100000:2.0:1789760777.002730:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756042853232 time 756081135358 state 0x0 ref 1 00000100:00100000:2.0:1789760777.002745:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756081395774 time 756081135358 state 0x1 ref 1 00000100:00100000:2.0:1789760777.002768:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756081395774 time 756107478458 state 0x0 ref 1 00000100:00100000:2.0:1789760777.002780:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508438656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.002835:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24092672, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:2.0:1789760777.002844:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe567e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.002859:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.002863:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.002870:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe567e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.002883:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe567e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760777.003403:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760777.030102:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973202 is committed 00002000:00010000:3.0:1789760777.030262:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24092672 -> 24096768 00010000:00010000:3.0:1789760777.030277:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe567e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.030366:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe567e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.030376:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe567e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.030383:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760777.030388:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe567e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.030398:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe567e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.030416:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe567e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.030424:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe567e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24092672->24096767] (req 24092672->24096767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.033702:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508438656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30914us (31139us total) trans 4294973202 rc 0/0 00000100:00100000:2.0:1789760777.033717:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5914 00000100:00100000:2.0:1789760777.033727:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756081395774 time 756107478458 state 0x2 ref 0 00000100:00100000:2.0:1789760777.038732:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760777.038845:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508438784 00000100:00100000:3.0:1789760777.038855:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508438784 00000100:00100000:3.0:1789760777.038908:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756081395774 time 756107478458 state 0x0 ref 1 00000100:00100000:3.0:1789760777.038932:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756107738874 time 756107478458 state 0x1 ref 1 00000100:00100000:3.0:1789760777.038999:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756107738874 time 756143709264 state 0x0 ref 1 00000100:00100000:3.0:1789760777.039013:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508438784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760777.039064:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24096768, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:3.0:1789760777.039076:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.039093:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760777.039097:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.039103:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.039118:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.069086:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973203 is committed 00002000:00010000:2.0:1789760777.069485:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24096768 -> 24100864 00010000:00010000:2.0:1789760777.069501:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.069513:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.069522:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.069530:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.069534:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.069543:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.069563:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.069570:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe567ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24096768->24100863] (req 24096768->24100863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.069661:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508438784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30646us (30935us total) trans 4294973203 rc 0/0 00000100:00100000:2.0:1789760777.069675:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5915 00000100:00100000:2.0:1789760777.069685:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756107738874 time 756143709264 state 0x2 ref 0 00000100:00100000:1.0:1789760777.079675:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760777.079827:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508438912 00000100:00100000:3.0:1789760777.079838:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508438912 00000100:00100000:3.0:1789760777.079895:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756107738874 time 756143709264 state 0x0 ref 1 00000100:00100000:3.0:1789760777.079910:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756143969680 time 756143709264 state 0x1 ref 1 00000100:00100000:3.0:1789760777.079943:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756143969680 time 756184653228 state 0x0 ref 1 00000100:00100000:3.0:1789760777.079955:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508438912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760777.080002:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24100864, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:3.0:1789760777.080011:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe567f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.080027:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760777.080031:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.080037:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe567f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.080050:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe567f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.088200:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7ef9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3475:1876699534084480:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088275:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088346:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499ca5dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534084992:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088376:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088395:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499ca5df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534085504:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088424:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088442:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534086016:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088466:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088494:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:0.0:1789760777.088499:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7efb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3475:1876699534084864:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088524:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088542:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499ca5c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534085376:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088562:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088578:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534085888:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088598:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088627:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7ef8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3472:1876699534084608:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088649:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088666:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499ca5d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3472:1876699534085120:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088686:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088703:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499ca5f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3472:1876699534085632:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088734:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088753:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:0.0:1789760777.088759:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7ef8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534084736:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088781:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088800:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499ca5c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3472:1876699534085248:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088830:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088846:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499ca5ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3472:1876699534085760:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.088864:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.088897:0:6460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534084480 00000100:00100000:0.0:1789760777.088905:0:6460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534084480 00000100:00100000:0.0:1789760777.088946:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 486 00000100:00100000:0.0:1789760777.088954:0:6460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cae050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3475:x1876699534084480:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089005:0:6460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cae050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3475:x1876699534084480:12345-0@lo:400:kworker.0 Request processed in 51us (730us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089011:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 486 00000100:00100000:0.0:1789760777.089041:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534084992 00000100:00100000:0.0:1789760777.089044:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534084992 00000100:00100000:0.0:1789760777.089065:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534084736 00000100:00100000:0.0:1789760777.089068:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534084736 00000100:00100000:0.0:1789760777.089083:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 458 00000100:00100000:0.0:1789760777.089089:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534084992:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089120:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534084992:12345-0@lo:400:kworker.0 Request processed in 31us (744us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089126:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 458 00000100:00100000:0.0:1789760777.089135:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 459 00000100:00100000:0.0:1789760777.089140:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534084736:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089167:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534084736:12345-0@lo:400:kworker.0 Request processed in 27us (385us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089172:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 459 00000100:00100000:0.0:1789760777.089188:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534085504 00000100:00100000:0.0:1789760777.089191:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534085504 00000100:00100000:0.0:1789760777.089215:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534086016 00000100:00100000:0.0:1789760777.089218:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534086016 00000100:00100000:0.0:1789760777.089228:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534085376 00000100:00100000:0.0:1789760777.089230:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534085376 00000100:00100000:0.0:1789760777.089240:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534085888 00000100:00100000:0.0:1789760777.089242:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534085888 00000100:00100000:0.0:1789760777.089254:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 243 00000100:00100000:0.0:1789760777.089259:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534085504:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089312:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534085504:12345-0@lo:400:kworker.0 Request processed in 53us (887us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089319:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 243 00000100:00100000:0.0:1789760777.089334:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 244 00000100:00100000:0.0:1789760777.089342:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534086016:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089368:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534086016:12345-0@lo:400:kworker.0 Request processed in 28us (902us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089374:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 244 00000100:00100000:0.0:1789760777.089385:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:0.0:1789760777.089391:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534085376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089421:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864f8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534085376:12345-0@lo:400:kworker.0 Request processed in 31us (858us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089427:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00100000:0.0:1789760777.089437:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:0.0:1789760777.089444:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534085888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089475:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534085888:12345-0@lo:400:kworker.0 Request processed in 32us (875us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089480:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00100000:0.0:1789760777.089502:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534084864 00000100:00100000:0.0:1789760777.089507:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534084864 00000100:00100000:0.0:1789760777.089526:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534084608 00000100:00100000:0.0:1789760777.089528:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534084608 00000100:00100000:0.0:1789760777.089538:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534085120 00000100:00100000:0.0:1789760777.089541:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534085120 00000100:00100000:0.0:1789760777.089550:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534085632 00000100:00100000:0.0:1789760777.089553:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534085632 00000100:00100000:0.0:1789760777.089562:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534085248 00000100:00100000:0.0:1789760777.089565:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534085248 00000100:00100000:0.0:1789760777.089575:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534085760 00000100:00100000:0.0:1789760777.089577:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534085760 00000100:00100000:0.0:1789760777.089591:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 963 00000100:00100000:0.0:1789760777.089596:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3475:x1876699534084864:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089624:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048864fb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3475:x1876699534084864:12345-0@lo:400:kworker.0 Request processed in 28us (1099us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089630:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 963 00000100:00100000:0.0:1789760777.089662:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 964 00000100:00100000:0.0:1789760777.089668:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3472:x1876699534084608:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089696:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3472:x1876699534084608:12345-0@lo:400:kworker.0 Request processed in 29us (1046us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089702:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 964 00000100:00100000:0.0:1789760777.089713:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 965 00000100:00100000:0.0:1789760777.089719:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3472:x1876699534085120:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089744:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd53100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3472:x1876699534085120:12345-0@lo:400:kworker.0 Request processed in 26us (1057us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089749:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 965 00000100:00100000:0.0:1789760777.089759:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 966 00000100:00100000:0.0:1789760777.089765:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3472:x1876699534085632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089790:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3472:x1876699534085632:12345-0@lo:400:kworker.0 Request processed in 25us (1055us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089795:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 966 00000100:00100000:0.0:1789760777.089804:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 967 00000100:00100000:0.0:1789760777.089819:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3472:x1876699534085248:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089843:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3472:x1876699534085248:12345-0@lo:400:kworker.0 Request processed in 35us (1013us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089848:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 967 00000100:00100000:0.0:1789760777.089858:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 968 00000100:00100000:0.0:1789760777.089863:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3472:x1876699534085760:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089882:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3472:x1876699534085760:12345-0@lo:400:kworker.0 Request processed in 19us (1017us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.089886:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 968 00000100:00080000:0.0:1789760777.089931:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789760777.089934:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760777.089937:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760777.089938:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760777.089964:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7ef8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534084736:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089974:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499ca5c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3472:1876699534085248:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089983:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499ca5ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3472:1876699534085760:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089992:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7ef8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3472:1876699534084608:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.089999:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499ca5d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3472:1876699534085120:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.090007:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499ca5f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3472:1876699534085632:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.090043:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7efb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3475:1876699534084864:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.090052:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499ca5c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534085376:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.090062:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534085888:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.090069:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7ef9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3475:1876699534084480:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.090078:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499ca5dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534084992:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.090086:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499ca5df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534085504:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.090094:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534086016:0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.091493:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1789760777.091512:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534086400:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760777.091570:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.091597:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760777.091602:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b7fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534086272:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760777.091610:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499cd52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534086528:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760777.091630:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.091649:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760777.091652:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534086144:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760777.091682:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.091690:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534086400 00000100:00100000:0.0:1789760777.091697:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534086400 00000100:00100000:3.0:1789760777.091828:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534086272 00000100:00100000:3.0:1789760777.091833:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534086272 00000100:00100000:3.0:1789760777.091857:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534086528 00000100:00100000:3.0:1789760777.091861:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534086528 00000100:00100000:3.0:1789760777.091874:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534086144 00000100:00100000:3.0:1789760777.091876:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534086144 00000100:00100000:3.0:1789760777.091890:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:3.0:1789760777.091897:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534086272:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1789760777.091917:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527499264 pending 1703936 free 3834642432 avail 3827302400 00000100:00100000:3.0:1789760777.091963:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888130000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534086272:12345-0@lo:13:osp-pre-0-1.0 Request processed in 67us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.091969:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:3.0:1789760777.091986:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:3.0:1789760777.091992:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534086528:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760777.092011:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760777.092043:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534086528:12345-0@lo:13:osp-pre-1-1.0 Request processed in 52us (395us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.092048:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:3.0:1789760777.092059:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:3.0:1789760777.092066:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534086144:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760777.092075:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527499264 pending 1703936 free 3834642432 avail 3827302400 00000100:00100000:3.0:1789760777.092097:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888133480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534086144:12345-0@lo:13:osp-pre-0-0.0 Request processed in 33us (415us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.092102:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:3.0:1789760777.092147:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499cd52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534086528:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760777.092276:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:0.0:1789760777.092308:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534086400:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760777.092323:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760777.092351:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888132300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534086400:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (783us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.092356:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:1.0:1789760777.092469:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534086400:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760777.092488:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b7fdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534086272:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760777.092505:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534086144:0@lo:13:osp-pre-0-0.0 00000001:00080000:3.0:1789760777.116843:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973204 is committed 00002000:00010000:2.0:1789760777.116915:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24100864 -> 24104960 00010000:00010000:2.0:1789760777.116933:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe567f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.116944:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe567f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.116952:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe567f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.116960:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.116966:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe567f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.116975:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe567f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.116994:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe567f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.117001:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe567f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24100864->24104959] (req 24100864->24104959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.117112:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508438912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37146us (37435us total) trans 4294973204 rc 0/0 00000100:00100000:2.0:1789760777.117122:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5916 00000100:00100000:2.0:1789760777.117131:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756143969680 time 756184653228 state 0x2 ref 0 00000100:00100000:1.0:1789760777.121239:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760777.121376:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508439040 00000100:00100000:2.0:1789760777.121397:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508439040 00000100:00100000:2.0:1789760777.121462:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756143969680 time 756184653228 state 0x0 ref 1 00000100:00100000:2.0:1789760777.121477:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756184913644 time 756184653228 state 0x1 ref 1 00000100:00100000:2.0:1789760777.121513:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756184913644 time 756226223758 state 0x0 ref 1 00000100:00100000:2.0:1789760777.121526:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888133100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508439040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.122035:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24104960, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:2.0:1789760777.122046:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe567fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.122062:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.122066:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.122073:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe567fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.122087:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe567fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.147727:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973205 is committed 00002000:00010000:2.0:1789760777.147779:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24104960 -> 24109056 00010000:00010000:2.0:1789760777.147794:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe567fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.147816:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe567fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.147826:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe567fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.147832:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.147837:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe567fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.147846:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe567fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.147866:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe567fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.147875:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe567fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24104960->24109055] (req 24104960->24109055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.148014:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888133100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508439040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26485us (26778us total) trans 4294973205 rc 0/0 00000100:00100000:2.0:1789760777.148028:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5917 00000100:00100000:2.0:1789760777.148038:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756184913644 time 756226223758 state 0x2 ref 0 00000100:00100000:1.0:1789760777.153585:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760777.153873:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508439168 00000100:00100000:2.0:1789760777.153882:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508439168 00000100:00100000:2.0:1789760777.153937:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756184913644 time 756226223758 state 0x0 ref 1 00000100:00100000:2.0:1789760777.153951:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756226484174 time 756226223758 state 0x1 ref 1 00000100:00100000:2.0:1789760777.154069:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756226484174 time 756258777472 state 0x0 ref 1 00000100:00100000:2.0:1789760777.154084:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508439168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.154131:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24109056, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:2.0:1789760777.154139:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56801 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.154166:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.154171:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.154178:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56801 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.154192:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56801 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760777.185610:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24109056 -> 24113152 00010000:00010000:2.0:1789760777.185630:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56801 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.185643:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56801 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.185652:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56801 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.185660:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.185664:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56801 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.185672:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56801 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.185692:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56801 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.185699:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56801 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24109056->24113151] (req 24109056->24113151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.185787:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973206 is committed 00000100:00100000:2.0:1789760777.185889:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508439168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31805us (32310us total) trans 4294973206 rc 0/0 00000100:00100000:2.0:1789760777.185897:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5918 00000100:00100000:2.0:1789760777.185904:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756226484174 time 756258777472 state 0x2 ref 0 00000100:00100000:1.0:1789760777.188620:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760777.188661:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508439296 00000100:00100000:2.0:1789760777.188671:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508439296 00000100:00100000:2.0:1789760777.188784:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756226484174 time 756258777472 state 0x0 ref 1 00000100:00100000:2.0:1789760777.188800:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756259037888 time 756258777472 state 0x1 ref 1 00000100:00100000:0.0:1789760777.188891:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756259037888 time 756293599349 state 0x0 ref 1 00000100:00100000:0.0:1789760777.188906:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508439296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760777.188954:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24113152, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:0.0:1789760777.188963:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56808 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.188980:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760777.188986:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.188993:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56808 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.189010:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56808 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.215973:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973207 is committed 00002000:00010000:2.0:1789760777.216460:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24113152 -> 24117248 00010000:00010000:2.0:1789760777.216475:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56808 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.216487:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56808 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.216496:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56808 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.216503:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.216508:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56808 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.216518:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56808 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.216538:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56808 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.216547:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56808 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24113152->24117247] (req 24113152->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.216682:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508439296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27776us (28064us total) trans 4294973207 rc 0/0 00000100:00100000:2.0:1789760777.216692:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5919 00000100:00100000:2.0:1789760777.216702:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756259037888 time 756293599349 state 0x2 ref 0 00000100:00100000:1.0:1789760777.220204:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760777.220593:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508439424 00000100:00100000:2.0:1789760777.220604:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508439424 00000100:00100000:2.0:1789760777.220658:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756259037888 time 756293599349 state 0x0 ref 1 00000100:00100000:2.0:1789760777.220675:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756293859765 time 756293599349 state 0x1 ref 1 00000100:00100000:2.0:1789760777.220696:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756293859765 time 756325406230 state 0x0 ref 1 00000100:00100000:2.0:1789760777.220708:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508439424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.220765:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24117248, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xb807 00010000:00010000:2.0:1789760777.220774:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5680f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.220790:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.220795:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.220801:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5680f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.220828:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5680f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.246403:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973208 is committed 00002000:00010000:2.0:1789760777.246672:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24117248 -> 24121344 00002000:00010000:2.0:1789760777.246678:0:8110:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:2.0:1789760777.246690:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5680f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.246703:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5680f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.246712:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5680f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.246719:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.246724:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5680f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.246734:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5680f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.246755:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5680f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.246762:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5680f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24117248->24121343] (req 24117248->24121343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.246862:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508439424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26151us (26663us total) trans 4294973208 rc 0/0 00000100:00100000:2.0:1789760777.246873:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5920 00000100:00100000:2.0:1789760777.246882:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756293859765 time 756325406230 state 0x2 ref 0 00000100:00100000:1.0:1789760777.257514:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760777.257632:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508439552 00000100:00100000:2.0:1789760777.257642:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508439552 00000100:00100000:2.0:1789760777.257816:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756293859765 time 756325406230 state 0x0 ref 1 00000100:00100000:2.0:1789760777.257834:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756325666646 time 756325406230 state 0x1 ref 1 00000100:00100000:2.0:1789760777.257867:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756325666646 time 756362577152 state 0x0 ref 1 00000100:00100000:2.0:1789760777.257880:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508439552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.257928:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24121344, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc007 00010000:00010000:2.0:1789760777.257937:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56816 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.257952:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.257955:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.257962:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56816 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.257976:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56816 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.296626:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973209 is committed 00002000:00010000:2.0:1789760777.296698:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24121344 -> 24125440 00010000:00010000:2.0:1789760777.296713:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56816 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.296725:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56816 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.296734:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56816 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.296741:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.296745:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56816 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.296754:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56816 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.296775:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56816 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.296782:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56816 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24121344->24125439] (req 24121344->24125439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.296879:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508439552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38997us (39369us total) trans 4294973209 rc 0/0 00000100:00100000:2.0:1789760777.296890:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5921 00000100:00100000:2.0:1789760777.296899:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756325666646 time 756362577152 state 0x2 ref 0 00000100:00100000:1.0:1789760777.300956:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760777.301062:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508439680 00000100:00100000:2.0:1789760777.301072:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508439680 00000100:00100000:2.0:1789760777.301123:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756325666646 time 756362577152 state 0x0 ref 1 00000100:00100000:2.0:1789760777.301137:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756362837568 time 756362577152 state 0x1 ref 1 00000100:00100000:2.0:1789760777.301212:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756362837568 time 756405921273 state 0x0 ref 1 00000100:00100000:2.0:1789760777.301231:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508439680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.301282:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24125440, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc007 00010000:00010000:2.0:1789760777.301394:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5681d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.301410:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.301414:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.301421:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5681d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.301435:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5681d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.330943:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973210 is committed 00002000:00010000:2.0:1789760777.331264:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24125440 -> 24129536 00010000:00010000:2.0:1789760777.331281:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5681d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.331312:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5681d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.331323:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5681d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.331331:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.331335:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5681d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.331345:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5681d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.331365:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5681d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.331373:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5681d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24125440->24129535] (req 24125440->24129535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.331500:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508439680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30271us (30555us total) trans 4294973210 rc 0/0 00000100:00100000:2.0:1789760777.331512:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5922 00000100:00100000:2.0:1789760777.331523:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756362837568 time 756405921273 state 0x2 ref 0 00000100:00100000:1.0:1789760777.337572:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760777.337660:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508439808 00000100:00100000:2.0:1789760777.337670:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508439808 00000100:00100000:2.0:1789760777.337721:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756362837568 time 756405921273 state 0x0 ref 1 00000100:00100000:2.0:1789760777.337737:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756406181689 time 756405921273 state 0x1 ref 1 00000100:00100000:2.0:1789760777.337763:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756406181689 time 756442473899 state 0x0 ref 1 00000100:00100000:2.0:1789760777.337775:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508439808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.337836:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24129536, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc007 00010000:00010000:2.0:1789760777.337853:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56824 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.337868:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.337873:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.337879:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56824 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.337893:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56824 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.371680:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973211 is committed 00002000:00010000:2.0:1789760777.371865:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24129536 -> 24133632 00010000:00010000:2.0:1789760777.371881:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56824 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.371894:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56824 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.371902:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56824 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.371910:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.371915:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56824 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.371924:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56824 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.371944:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56824 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.371952:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56824 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24129536->24133631] (req 24129536->24133631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.372049:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508439808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34270us (34481us total) trans 4294973211 rc 0/0 00000100:00100000:2.0:1789760777.372060:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5923 00000100:00100000:2.0:1789760777.372070:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756406181689 time 756442473899 state 0x2 ref 0 00000100:00100000:1.0:1789760777.376538:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760777.376663:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508439936 00000100:00100000:2.0:1789760777.376673:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508439936 00000100:00100000:2.0:1789760777.376737:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756406181689 time 756442473899 state 0x0 ref 1 00000100:00100000:2.0:1789760777.376753:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756442734315 time 756442473899 state 0x1 ref 1 00000100:00100000:2.0:1789760777.376788:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756442734315 time 756481498704 state 0x0 ref 1 00000100:00100000:2.0:1789760777.376817:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508439936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.376866:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24133632, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc007 00010000:00010000:2.0:1789760777.376875:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5682b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.376892:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.376896:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.376902:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5682b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.376916:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5682b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.403865:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973212 is committed 00002000:00010000:3.0:1789760777.403962:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24133632 -> 24137728 00010000:00010000:3.0:1789760777.403976:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5682b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.403988:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5682b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.403996:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5682b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.404003:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760777.404007:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5682b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.404017:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5682b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.404037:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5682b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.404045:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5682b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24133632->24137727] (req 24133632->24137727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.404163:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508439936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27346us (27624us total) trans 4294973212 rc 0/0 00000100:00100000:3.0:1789760777.404173:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5924 00000100:00100000:3.0:1789760777.404181:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756442734315 time 756481498704 state 0x2 ref 0 00000100:00100000:0.0:1789760777.409710:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760777.409830:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508440064 00000100:00100000:2.0:1789760777.409852:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508440064 00000100:00100000:2.0:1789760777.409906:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756442734315 time 756481498704 state 0x0 ref 1 00000100:00100000:2.0:1789760777.409922:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756481759120 time 756481498704 state 0x1 ref 1 00000100:00100000:2.0:1789760777.409990:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756481759120 time 756514699481 state 0x0 ref 1 00000100:00100000:2.0:1789760777.410004:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508440064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.410052:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24137728, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc007 00010000:00010000:2.0:1789760777.410060:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56832 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.410076:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.410080:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.410086:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56832 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.410100:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56832 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.440069:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973213 is committed 00002000:00010000:2.0:1789760777.441394:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24137728 -> 24141824 00010000:00010000:2.0:1789760777.441411:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56832 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.441424:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56832 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.441434:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56832 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.441442:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.441447:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56832 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.441457:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56832 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.441477:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56832 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.441486:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56832 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24137728->24141823] (req 24137728->24141823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.443789:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508440064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33774us (34075us total) trans 4294973213 rc 0/0 00000100:00100000:3.0:1789760777.443818:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5925 00000100:00100000:3.0:1789760777.443828:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756481759120 time 756514699481 state 0x2 ref 0 00000100:00100000:1.0:1789760777.451233:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760777.451424:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508440192 00000100:00100000:2.0:1789760777.451435:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508440192 00000100:00100000:2.0:1789760777.451487:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756481759120 time 756514699481 state 0x0 ref 1 00000100:00100000:2.0:1789760777.451503:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756514959897 time 756514699481 state 0x1 ref 1 00000100:00100000:2.0:1789760777.451535:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756514959897 time 756556245121 state 0x0 ref 1 00000100:00100000:2.0:1789760777.451547:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508440192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.451592:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24141824, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc007 00010000:00010000:2.0:1789760777.451600:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56839 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.451615:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.451620:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.451626:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56839 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.451639:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56839 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.478897:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973214 is committed 00002000:00010000:2.0:1789760777.479026:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24141824 -> 24145920 00010000:00010000:2.0:1789760777.479041:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56839 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.479060:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56839 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.479069:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56839 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.479075:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.479080:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56839 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.479090:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56839 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.479109:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56839 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.479117:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56839 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24141824->24145919] (req 24141824->24145919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.479370:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508440192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27716us (28038us total) trans 4294973214 rc 0/0 00000100:00100000:2.0:1789760777.479383:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5926 00000100:00100000:2.0:1789760777.479393:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756514959897 time 756556245121 state 0x2 ref 0 00000100:00100000:1.0:1789760777.497104:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760777.497244:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508440320 00000100:00100000:3.0:1789760777.497254:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508440320 00000100:00100000:3.0:1789760777.497341:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756514959897 time 756556245121 state 0x0 ref 1 00000100:00100000:3.0:1789760777.497356:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756556505537 time 756556245121 state 0x1 ref 1 00000100:00100000:3.0:1789760777.497389:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756556505537 time 756602099778 state 0x0 ref 1 00000100:00100000:3.0:1789760777.497402:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508440320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760777.497454:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24145920, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0xc007 00010000:00010000:3.0:1789760777.497464:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56840 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.497480:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760777.497484:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.497492:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56840 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.497507:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56840 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.545463:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973215 is committed 00002000:00010000:3.0:1789760777.545629:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24145920 -> 24150016 00002000:00010000:3.0:1789760777.545635:0:14638:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760777 -> 1789760778 00002000:00010000:3.0:1789760777.545638:0:14638:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760777 -> 1789760778 00010000:00010000:3.0:1789760777.545651:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56840 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.545664:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56840 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.545673:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56840 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.545684:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760777.545689:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56840 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.545697:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56840 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.545714:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56840 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.545798:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56840 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24145920->24150015] (req 24145920->24150015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.545931:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508440320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48526us (48831us total) trans 4294973215 rc 0/0 00000100:00100000:3.0:1789760777.545942:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5927 00000100:00100000:3.0:1789760777.545952:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756556505537 time 756602099778 state 0x2 ref 0 00000100:00100000:0.0:1789760777.549382:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760777.550510:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508440448 00000100:00100000:3.0:1789760777.550522:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508440448 00000100:00100000:2.0:1789760777.550683:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756556505537 time 756602099778 state 0x0 ref 1 00000100:00100000:2.0:1789760777.550701:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756602360194 time 756602099778 state 0x1 ref 1 00000100:00100000:2.0:1789760777.550851:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756602360194 time 756655559644 state 0x0 ref 1 00000100:00100000:2.0:1789760777.550867:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508440448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.550915:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24150016, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:2.0:1789760777.550924:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56847 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.550940:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.550945:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.550952:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56847 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.550967:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56847 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.582440:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973216 is committed 00002000:00010000:3.0:1789760777.582528:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24150016 -> 24154112 00010000:00010000:3.0:1789760777.582542:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56847 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.582555:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56847 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.582569:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56847 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.582576:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760777.582580:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56847 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.582588:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56847 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.582607:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56847 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.582615:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56847 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24150016->24154111] (req 24150016->24154111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.582708:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508440448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31840us (33330us total) trans 4294973216 rc 0/0 00000100:00100000:3.0:1789760777.582790:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5928 00000100:00100000:3.0:1789760777.582801:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756602360194 time 756655559644 state 0x2 ref 0 00000100:00100000:1.0:1789760777.587237:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760777.587508:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508440576 00000100:00100000:2.0:1789760777.587518:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508440576 00000100:00100000:2.0:1789760777.587571:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756602360194 time 756655559644 state 0x0 ref 1 00000100:00100000:2.0:1789760777.587587:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756655820060 time 756655559644 state 0x1 ref 1 00000100:00100000:2.0:1789760777.587617:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756655820060 time 756692327700 state 0x0 ref 1 00000100:00100000:2.0:1789760777.587629:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888132a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508440576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.587677:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24154112, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:2.0:1789760777.587686:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5684e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.587702:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.587707:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.587713:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5684e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.587828:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5684e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.614882:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973217 is committed 00002000:00010000:2.0:1789760777.616089:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24154112 -> 24158208 00010000:00010000:2.0:1789760777.616106:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5684e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.616119:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5684e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.616129:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5684e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.616137:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.616142:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5684e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.616151:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5684e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.616172:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5684e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.616180:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe5684e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24154112->24158207] (req 24154112->24158207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.616272:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888132a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508440576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28639us (29039us total) trans 4294973217 rc 0/0 00000100:00100000:2.0:1789760777.616281:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5929 00000100:00100000:2.0:1789760777.616336:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756655820060 time 756692327700 state 0x2 ref 0 00000100:00100000:2.0:1789760777.619905:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760777.620342:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508440704 00000100:00100000:3.0:1789760777.620352:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508440704 00000100:00100000:3.0:1789760777.620417:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756655820060 time 756692327700 state 0x0 ref 1 00000100:00100000:3.0:1789760777.620439:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756692588116 time 756692327700 state 0x1 ref 1 00000100:00100000:3.0:1789760777.620465:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756692588116 time 756725175182 state 0x0 ref 1 00000100:00100000:3.0:1789760777.620478:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508440704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760777.620526:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24158208, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:3.0:1789760777.620535:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56855 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.620551:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760777.620556:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.620563:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56855 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.620577:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56855 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.675708:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973218 is committed 00002000:00010000:0.0:1789760777.675980:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24158208 -> 24162304 00010000:00010000:0.0:1789760777.675995:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56855 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.676007:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56855 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.676016:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56855 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.676023:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760777.676028:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56855 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.676042:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56855 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.676062:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56855 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.676071:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56855 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24158208->24162303] (req 24158208->24162303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.676177:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508440704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55697us (56288us total) trans 4294973218 rc 0/0 00000100:00100000:0.0:1789760777.676188:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5930 00000100:00100000:0.0:1789760777.676198:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756692588116 time 756725175182 state 0x2 ref 0 00000100:00100000:0.0:1789760777.684789:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760777.690829:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508440832 00000100:00100000:3.0:1789760777.690846:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508440832 00000100:00100000:2.0:1789760777.693825:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756692588116 time 756725175182 state 0x0 ref 1 00000100:00100000:2.0:1789760777.693845:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756725435598 time 756725175182 state 0x1 ref 1 00000100:00100000:2.0:1789760777.694851:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756725435598 time 756799555298 state 0x0 ref 1 00000100:00100000:2.0:1789760777.694875:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508440832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.694929:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24162304, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:2.0:1789760777.694938:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5685c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.694953:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:2.0:1789760777.694958:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.694964:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5685c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.694978:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5685c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760777.731614:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24162304 -> 24166400 00010000:00010000:3.0:1789760777.731636:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5685c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.731650:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5685c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.731665:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5685c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.731673:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760777.731677:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5685c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.731687:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5685c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.731707:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5685c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.731716:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe5685c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24162304->24166399] (req 24162304->24166399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.731824:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508440832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36953us (47039us total) trans 4294973219 rc 0/0 00000100:00100000:3.0:1789760777.731835:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5931 00000100:00100000:3.0:1789760777.731846:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756725435598 time 756799555298 state 0x2 ref 0 00000001:00080000:1.0:1789760777.732321:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973219 is committed 00000100:00100000:2.0:1789760777.734885:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760777.734989:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508440960 00000100:00100000:0.0:1789760777.734998:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508440960 00000100:00100000:0.0:1789760777.735055:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756725435598 time 756799555298 state 0x0 ref 1 00000100:00100000:0.0:1789760777.735072:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756799815714 time 756799555298 state 0x1 ref 1 00000100:00100000:0.0:1789760777.735132:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756799815714 time 756839841909 state 0x0 ref 1 00000100:00100000:0.0:1789760777.735145:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7343b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508440960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760777.735249:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24166400, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:0.0:1789760777.735260:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56863 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.735275:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760777.735279:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.735307:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56863 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.735322:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56863 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.775935:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973220 is committed 00002000:00010000:0.0:1789760777.776039:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24166400 -> 24170496 00010000:00010000:0.0:1789760777.776055:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56863 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.776066:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56863 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.776075:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56863 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.776082:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760777.776087:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56863 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.776096:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56863 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.776121:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56863 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.776129:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56863 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24166400->24170495] (req 24166400->24170495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.776407:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7343b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508440960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41258us (41524us total) trans 4294973220 rc 0/0 00000100:00100000:0.0:1789760777.776418:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5932 00000100:00100000:0.0:1789760777.776428:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756799815714 time 756839841909 state 0x2 ref 0 00000100:00100000:1.0:1789760777.780929:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760777.781046:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508441088 00000100:00100000:0.0:1789760777.781054:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508441088 00000100:00100000:0.0:1789760777.781101:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756799815714 time 756839841909 state 0x0 ref 1 00000100:00100000:0.0:1789760777.781111:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756840102325 time 756839841909 state 0x1 ref 1 00000100:00100000:0.0:1789760777.781134:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756840102325 time 756885844258 state 0x0 ref 1 00000100:00100000:0.0:1789760777.781146:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508441088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760777.781197:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24170496, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:0.0:1789760777.781206:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5686a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.781221:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760777.781226:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.781232:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5686a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.781246:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5686a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.806531:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973221 is committed 00002000:00010000:3.0:1789760777.806655:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24170496 -> 24174592 00010000:00010000:3.0:1789760777.806669:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5686a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.806688:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5686a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.806697:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5686a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.806839:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760777.806849:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5686a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.806859:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5686a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.806879:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5686a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.806887:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5686a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24170496->24174591] (req 24170496->24174591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.807016:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508441088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25866us (26091us total) trans 4294973221 rc 0/0 00000100:00100000:3.0:1789760777.807026:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5933 00000100:00100000:3.0:1789760777.807036:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756840102325 time 756885844258 state 0x2 ref 0 00000100:00100000:1.0:1789760777.810014:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760777.810107:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508441216 00000100:00100000:3.0:1789760777.810119:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508441216 00000100:00100000:3.0:1789760777.810203:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756840102325 time 756885844258 state 0x0 ref 1 00000100:00100000:3.0:1789760777.810218:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756886104674 time 756885844258 state 0x1 ref 1 00000100:00100000:3.0:1789760777.810252:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756886104674 time 756914962215 state 0x0 ref 1 00000100:00100000:3.0:1789760777.810265:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508441216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760777.810342:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24174592, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:3.0:1789760777.810352:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56871 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.810367:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760777.810372:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.810378:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56871 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.810392:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56871 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.841900:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973222 is committed 00002000:00010000:3.0:1789760777.842070:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24174592 -> 24178688 00010000:00010000:3.0:1789760777.842086:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56871 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.842098:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56871 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.842112:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56871 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.842119:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760777.842124:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56871 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.842133:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56871 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.842252:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56871 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.842261:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56871 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24174592->24178687] (req 24174592->24178687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.842416:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508441216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32149us (32407us total) trans 4294973222 rc 0/0 00000100:00100000:3.0:1789760777.842426:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5934 00000100:00100000:3.0:1789760777.842436:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756886104674 time 756914962215 state 0x2 ref 0 00000100:00100000:2.0:1789760777.846511:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760777.846616:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508441344 00000100:00100000:0.0:1789760777.846627:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508441344 00000100:00100000:0.0:1789760777.846682:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756886104674 time 756914962215 state 0x0 ref 1 00000100:00100000:0.0:1789760777.846698:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756915222631 time 756914962215 state 0x1 ref 1 00000100:00100000:0.0:1789760777.846737:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756915222631 time 756951439010 state 0x0 ref 1 00000100:00100000:0.0:1789760777.846750:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508441344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760777.846811:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24178688, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:0.0:1789760777.846821:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56878 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.846837:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:0.0:1789760777.846841:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.846848:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56878 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.846862:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56878 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.879602:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973223 is committed 00002000:00010000:2.0:1789760777.879699:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24178688 -> 24182784 00010000:00010000:2.0:1789760777.879715:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56878 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.879728:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56878 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.879737:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56878 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.879744:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760777.879749:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56878 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.879758:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56878 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.879783:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56878 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.879792:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56878 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24178688->24182783] (req 24178688->24182783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.880730:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508441344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33538us (33786us total) trans 4294973223 rc 0/0 00000100:00100000:2.0:1789760777.880742:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5935 00000100:00100000:2.0:1789760777.880751:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756915222631 time 756951439010 state 0x2 ref 0 00000100:00100000:2.0:1789760777.886149:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760777.886312:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508441472 00000100:00100000:3.0:1789760777.886322:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508441472 00000100:00100000:3.0:1789760777.886372:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756915222631 time 756951439010 state 0x0 ref 1 00000100:00100000:3.0:1789760777.886390:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756951699426 time 756951439010 state 0x1 ref 1 00000100:00100000:3.0:1789760777.886419:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756951699426 time 756991130537 state 0x0 ref 1 00000100:00100000:3.0:1789760777.886431:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508441472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760777.886477:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24182784, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:3.0:1789760777.886486:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5687f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.886501:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=756s v=(0 0) 00010000:00010000:3.0:1789760777.886505:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.886512:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5687f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.886525:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5687f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760777.915963:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24182784 -> 24186880 00010000:00010000:3.0:1789760777.915982:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5687f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.915994:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5687f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.916003:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5687f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.916016:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760777.916021:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5687f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.916031:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5687f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.916054:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5687f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.916063:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe5687f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24182784->24186879] (req 24182784->24186879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.916196:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508441472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29760us (30051us total) trans 4294973224 rc 0/0 00000100:00100000:3.0:1789760777.916207:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5936 00000100:00100000:3.0:1789760777.916217:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756951699426 time 756991130537 state 0x2 ref 0 00000001:00080000:3.0:1789760777.916256:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973224 is committed 00000100:00100000:1.0:1789760777.919616:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760777.919693:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508441600 00000100:00100000:3.0:1789760777.919703:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508441600 00000100:00100000:3.0:1789760777.919850:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756951699426 time 756991130537 state 0x0 ref 1 00000100:00100000:3.0:1789760777.919866:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756991390953 time 756991130537 state 0x1 ref 1 00000100:00100000:3.0:1789760777.919901:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756991390953 time 757024611361 state 0x0 ref 1 00000100:00100000:3.0:1789760777.919915:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508441600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760777.919962:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24186880, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:3.0:1789760777.919970:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56886 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.919986:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760777.919990:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.919997:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56886 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.920011:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56886 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.950532:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973225 is committed 00002000:00010000:3.0:1789760777.950672:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24186880 -> 24190976 00010000:00010000:3.0:1789760777.950687:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56886 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.950700:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56886 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.950709:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56886 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.950733:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760777.950739:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56886 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.950749:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56886 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.950770:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56886 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.950779:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56886 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24186880->24190975] (req 24186880->24190975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.950876:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508441600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30959us (31265us total) trans 4294973225 rc 0/0 00000100:00100000:3.0:1789760777.950886:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5937 00000100:00100000:3.0:1789760777.950895:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756991390953 time 757024611361 state 0x2 ref 0 00000100:00100000:2.0:1789760777.955036:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760777.955175:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508441728 00000100:00100000:3.0:1789760777.955185:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508441728 00000100:00100000:3.0:1789760777.955241:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 756991390953 time 757024611361 state 0x0 ref 1 00000100:00100000:3.0:1789760777.955256:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757024871777 time 757024611361 state 0x1 ref 1 00000100:00100000:1.0:1789760777.955426:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757024871777 time 757060134036 state 0x0 ref 1 00000100:00100000:1.0:1789760777.955441:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508441728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760777.955491:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24190976, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:1.0:1789760777.955499:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5688d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.955516:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760777.955521:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.955528:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5688d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.955542:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5688d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.983994:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760777.988062:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24190976 -> 24195072 00010000:00010000:1.0:1789760777.988081:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5688d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.988094:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5688d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.988103:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5688d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.988109:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760777.988114:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5688d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.988123:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5688d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.988142:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5688d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.988149:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5688d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24190976->24195071] (req 24190976->24195071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.988256:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508441728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32813us (33226us total) trans 4294973226 rc 0/0 00000100:00100000:1.0:1789760777.988267:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5938 00000100:00100000:1.0:1789760777.988276:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757024871777 time 757060134036 state 0x2 ref 0 00000001:00080000:3.0:1789760777.988891:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973226 is committed 00000100:00100000:2.0:1789760777.997274:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760777.997347:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508441856 00000100:00100000:3.0:1789760777.997357:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508441856 00000100:00100000:3.0:1789760777.997410:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757024871777 time 757060134036 state 0x0 ref 1 00000100:00100000:3.0:1789760777.997424:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757060394452 time 757060134036 state 0x1 ref 1 00000100:00100000:2.0:1789760777.997820:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757060394452 time 757102525589 state 0x0 ref 1 00000100:00100000:2.0:1789760777.997835:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508441856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760777.997885:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24195072, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:2.0:1789760777.997894:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56894 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.997909:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760777.997914:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.997920:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56894 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.997935:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56894 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.033590:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973227 is committed 00002000:00010000:3.0:1789760778.033743:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24195072 -> 24199168 00010000:00010000:3.0:1789760778.033761:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56894 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.033774:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56894 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.033783:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56894 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.033790:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760778.033795:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56894 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.033813:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56894 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.033833:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56894 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.033840:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe56894 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24195072->24199167] (req 24195072->24199167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.033930:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508441856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36094us (36662us total) trans 4294973227 rc 0/0 00000100:00100000:3.0:1789760778.033940:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5939 00000100:00100000:3.0:1789760778.033950:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757060394452 time 757102525589 state 0x2 ref 0 00000100:00100000:2.0:1789760778.037554:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760778.037659:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508441984 00000100:00100000:3.0:1789760778.037669:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508441984 00000100:00100000:3.0:1789760778.037741:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757060394452 time 757102525589 state 0x0 ref 1 00000100:00100000:3.0:1789760778.037757:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757102786005 time 757102525589 state 0x1 ref 1 00000100:00100000:3.0:1789760778.037793:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757102786005 time 757142502713 state 0x0 ref 1 00000100:00100000:3.0:1789760778.037827:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508441984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760778.037887:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24199168, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:3.0:1789760778.037897:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5689b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.037913:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760778.037918:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.037924:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5689b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.037940:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5689b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.081065:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973228 is committed 00002000:00010000:3.0:1789760778.081305:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24199168 -> 24203264 00010000:00010000:3.0:1789760778.081320:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5689b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.081337:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5689b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.081346:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5689b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.081352:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760778.081357:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5689b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.081365:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5689b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.081383:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5689b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.081389:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5689b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24199168->24203263] (req 24199168->24203263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.081513:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508441984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43685us (43962us total) trans 4294973228 rc 0/0 00000100:00100000:3.0:1789760778.081523:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5940 00000100:00100000:3.0:1789760778.081533:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757102786005 time 757142502713 state 0x2 ref 0 00000100:00100000:2.0:1789760778.084636:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760778.084730:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508442112 00000100:00100000:1.0:1789760778.084740:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508442112 00000100:00100000:1.0:1789760778.084792:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757102786005 time 757142502713 state 0x0 ref 1 00000100:00100000:1.0:1789760778.084819:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757142763129 time 757142502713 state 0x1 ref 1 00000100:00100000:1.0:1789760778.084880:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757142763129 time 757189590595 state 0x0 ref 1 00000100:00100000:1.0:1789760778.084894:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508442112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760778.084938:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24203264, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:1.0:1789760778.084947:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe568a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.084963:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760778.084967:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.084973:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe568a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.084987:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe568a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.110492:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760778.110515:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499ca5df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534086656:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760778.110597:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760778.110742:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534086656 00000100:00100000:0.0:1789760778.110752:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534086656 00000100:00100000:0.0:1789760778.110816:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 460 00000100:00100000:0.0:1789760778.110826:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534086656:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760778.110864:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789760778.110913:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafa300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534086656:12345-0@lo:41:osp-pre-0-1.0 Request processed in 89us (316us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760778.110921:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 460 00000100:00100000:0.0:1789760778.110976:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499ca5df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534086656:0@lo:41:osp-pre-0-1.0 00000001:00080000:2.0:1789760778.116417:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973229 is committed 00002000:00010000:1.0:1789760778.117096:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24203264 -> 24207360 00010000:00010000:1.0:1789760778.117113:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe568a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.117123:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe568a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.117129:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe568a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.117133:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760778.117137:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe568a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.117143:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe568a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.117159:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe568a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.117165:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe568a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24203264->24207359] (req 24203264->24207359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.117257:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508442112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32360us (32625us total) trans 4294973229 rc 0/0 00000100:00100000:1.0:1789760778.117267:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5941 00000100:00100000:1.0:1789760778.117276:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757142763129 time 757189590595 state 0x2 ref 0 00000100:00100000:2.0:1789760778.125874:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760778.125927:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508442240 00000100:00100000:3.0:1789760778.125937:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508442240 00000100:00100000:3.0:1789760778.126057:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757142763129 time 757189590595 state 0x0 ref 1 00000100:00100000:3.0:1789760778.126076:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757189851011 time 757189590595 state 0x1 ref 1 00000100:00100000:3.0:1789760778.126108:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757189851011 time 757230818327 state 0x0 ref 1 00000100:00100000:3.0:1789760778.126120:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508442240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760778.126185:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24207360, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:3.0:1789760778.126838:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.126860:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760778.126865:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.126872:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.126887:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760778.163372:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24207360 -> 24211456 00010000:00010000:0.0:1789760778.163393:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.163407:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.163415:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.163422:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760778.163427:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.163441:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.163460:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.163468:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24207360->24211455] (req 24207360->24211455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.163573:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508442240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37450us (37712us total) trans 4294973230 rc 0/0 00000100:00100000:0.0:1789760778.163583:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5942 00000100:00100000:0.0:1789760778.163592:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757189851011 time 757230818327 state 0x2 ref 0 00000001:00080000:1.0:1789760778.164470:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973230 is committed 00000100:00100000:2.0:1789760778.170779:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760778.174120:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508442368 00000100:00100000:3.0:1789760778.174135:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508442368 00000100:00100000:3.0:1789760778.174200:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757189851011 time 757230818327 state 0x0 ref 1 00000100:00100000:3.0:1789760778.174216:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757231078743 time 757230818327 state 0x1 ref 1 00000100:00100000:3.0:1789760778.174341:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757231078743 time 757279050461 state 0x0 ref 1 00000100:00100000:3.0:1789760778.174361:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508442368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760778.174412:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24211456, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:3.0:1789760778.174421:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.174438:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760778.174443:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.174449:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.174464:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.210650:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973231 is committed 00002000:00010000:3.0:1789760778.210763:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24211456 -> 24215552 00010000:00010000:3.0:1789760778.210779:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.210793:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.210813:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.210821:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760778.210826:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.210836:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.210856:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.210865:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe568b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24211456->24215551] (req 24211456->24215551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.210960:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508442368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36598us (40188us total) trans 4294973231 rc 0/0 00000100:00100000:3.0:1789760778.210970:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5943 00000100:00100000:3.0:1789760778.210979:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757231078743 time 757279050461 state 0x2 ref 0 00000100:00100000:3.0:1789760778.216484:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760778.218360:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508442496 00000100:00100000:1.0:1789760778.218374:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508442496 00000100:00100000:2.0:1789760778.220794:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757231078743 time 757279050461 state 0x0 ref 1 00000100:00100000:2.0:1789760778.220824:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757279310877 time 757279050461 state 0x1 ref 1 00000100:00100000:2.0:1789760778.220914:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757279310877 time 757325623317 state 0x0 ref 1 00000100:00100000:2.0:1789760778.220930:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508442496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760778.220981:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24215552, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:2.0:1789760778.220990:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe568b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.221006:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760778.221011:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.221018:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe568b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.221033:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe568b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760778.277096:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24215552 -> 24219648 00010000:00010000:0.0:1789760778.277117:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe568b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.277131:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe568b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.277140:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe568b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.277154:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760778.277159:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe568b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.277167:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe568b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.277187:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe568b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.277196:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe568b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24215552->24219647] (req 24215552->24219647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.277328:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508442496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56397us (60850us total) trans 4294973232 rc 0/0 00000100:00100000:0.0:1789760778.277339:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5944 00000100:00100000:0.0:1789760778.277349:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757279310877 time 757325623317 state 0x2 ref 0 00000001:00080000:3.0:1789760778.277811:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973232 is committed 00000100:00100000:1.0:1789760778.286508:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760778.286858:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508442624 00000100:00100000:0.0:1789760778.286869:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508442624 00000100:00100000:0.0:1789760778.286926:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757279310877 time 757325623317 state 0x0 ref 1 00000100:00100000:0.0:1789760778.286942:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757325883733 time 757325623317 state 0x1 ref 1 00000100:00100000:0.0:1789760778.286973:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757325883733 time 757391683268 state 0x0 ref 1 00000100:00100000:0.0:1789760778.286990:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508442624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760778.287038:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24219648, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:0.0:1789760778.287047:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.287062:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760778.287067:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.287074:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.287088:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.318971:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973233 is committed 00002000:00010000:0.0:1789760778.319061:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24219648 -> 24223744 00010000:00010000:0.0:1789760778.319075:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.319088:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.319097:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.319104:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760778.319109:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.319120:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.319139:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.319202:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24219648->24223743] (req 24219648->24223743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.319342:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508442624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32350us (32839us total) trans 4294973233 rc 0/0 00000100:00100000:0.0:1789760778.319352:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5945 00000100:00100000:0.0:1789760778.319361:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757325883733 time 757391683268 state 0x2 ref 0 00000100:00100000:3.0:1789760778.322425:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760778.322473:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508442752 00000100:00100000:0.0:1789760778.322484:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508442752 00000100:00100000:0.0:1789760778.322541:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757325883733 time 757391683268 state 0x0 ref 1 00000100:00100000:0.0:1789760778.322555:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757391943684 time 757391683268 state 0x1 ref 1 00000100:00100000:0.0:1789760778.322894:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757391943684 time 757427548359 state 0x0 ref 1 00000100:00100000:0.0:1789760778.322912:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508442752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760778.322960:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24223744, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:0.0:1789760778.322969:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe568c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.322986:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760778.322990:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.322996:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe568c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.323011:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe568c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760778.354130:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24223744 -> 24227840 00010000:00010000:3.0:1789760778.354155:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe568c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.354170:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe568c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.354180:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe568c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.354188:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760778.354194:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe568c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.354204:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe568c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.354228:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe568c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.354236:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe568c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24223744->24227839] (req 24223744->24227839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.354392:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508442752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31476us (31967us total) trans 4294973234 rc 0/0 00000100:00100000:3.0:1789760778.354405:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5946 00000100:00100000:3.0:1789760778.354417:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757391943684 time 757427548359 state 0x2 ref 0 00000001:00080000:3.0:1789760778.354465:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973234 is committed 00000100:00100000:1.0:1789760778.358570:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760778.358713:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508442880 00000100:00100000:0.0:1789760778.358722:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508442880 00000100:00100000:0.0:1789760778.358779:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757391943684 time 757427548359 state 0x0 ref 1 00000100:00100000:0.0:1789760778.358795:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757427808775 time 757427548359 state 0x1 ref 1 00000100:00100000:0.0:1789760778.358844:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757427808775 time 757463553788 state 0x0 ref 1 00000100:00100000:0.0:1789760778.358857:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508442880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760778.358916:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24227840, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:0.0:1789760778.358926:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.358940:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760778.358945:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.358952:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.358973:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.387931:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973235 is committed 00002000:00010000:1.0:1789760778.388035:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24227840 -> 24231936 00010000:00010000:1.0:1789760778.388053:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.388066:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.388075:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.388083:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760778.388088:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.388097:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.388119:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.388126:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe568cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24227840->24231935] (req 24227840->24231935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.388229:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508442880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29369us (29664us total) trans 4294973235 rc 0/0 00000100:00100000:1.0:1789760778.388240:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5947 00000100:00100000:1.0:1789760778.388249:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757427808775 time 757463553788 state 0x2 ref 0 00000100:00100000:2.0:1789760778.392559:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760778.392654:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508443008 00000100:00100000:0.0:1789760778.392665:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508443008 00000100:00100000:0.0:1789760778.392733:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757427808775 time 757463553788 state 0x0 ref 1 00000100:00100000:0.0:1789760778.392750:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757463814204 time 757463553788 state 0x1 ref 1 00000100:00100000:0.0:1789760778.392777:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757463814204 time 757497487666 state 0x0 ref 1 00000100:00100000:0.0:1789760778.392790:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508443008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760778.392849:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24231936, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:0.0:1789760778.392859:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.392874:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760778.392878:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.392884:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.392899:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.419175:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973236 is committed 00002000:00010000:3.0:1789760778.419492:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24231936 -> 24236032 00010000:00010000:3.0:1789760778.419511:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.419526:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.419536:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.419544:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760778.419549:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.419559:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.419582:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.419591:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24231936->24236031] (req 24231936->24236031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.419743:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508443008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26950us (27189us total) trans 4294973236 rc 0/0 00000100:00100000:3.0:1789760778.419754:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5948 00000100:00100000:3.0:1789760778.419765:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757463814204 time 757497487666 state 0x2 ref 0 00000100:00100000:3.0:1789760778.423521:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760778.423658:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508443136 00000100:00100000:0.0:1789760778.423669:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508443136 00000100:00100000:0.0:1789760778.423957:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757463814204 time 757497487666 state 0x0 ref 1 00000100:00100000:0.0:1789760778.423973:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757497748082 time 757497487666 state 0x1 ref 1 00000100:00100000:0.0:1789760778.424058:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757497748082 time 757528767025 state 0x0 ref 1 00000100:00100000:0.0:1789760778.424074:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508443136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760778.424126:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24236032, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:0.0:1789760778.424136:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.424152:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760778.424157:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.424163:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.424176:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.447791:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973237 is committed 00002000:00010000:1.0:1789760778.447880:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24236032 -> 24240128 00010000:00010000:1.0:1789760778.447896:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.447907:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.447916:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.447923:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760778.447928:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.447935:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.447954:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.447962:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24236032->24240127] (req 24236032->24240127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.448066:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508443136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23991us (24547us total) trans 4294973237 rc 0/0 00000100:00100000:1.0:1789760778.448077:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5949 00000100:00100000:1.0:1789760778.448087:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757497748082 time 757528767025 state 0x2 ref 0 00000100:00100000:1.0:1789760778.452956:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760778.453512:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508443264 00000100:00100000:0.0:1789760778.453523:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508443264 00000100:00100000:0.0:1789760778.453582:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757497748082 time 757528767025 state 0x0 ref 1 00000100:00100000:0.0:1789760778.453597:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757529027441 time 757528767025 state 0x1 ref 1 00000100:00100000:0.0:1789760778.453632:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757529027441 time 757558341290 state 0x0 ref 1 00000100:00100000:0.0:1789760778.453653:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508443264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760778.453712:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24240128, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:0.0:1789760778.453721:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.453737:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760778.453742:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.453747:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.453761:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.478018:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973238 is committed 00002000:00010000:0.0:1789760778.478129:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24240128 -> 24244224 00010000:00010000:0.0:1789760778.478283:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.478317:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.478326:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.478501:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760778.478506:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.478516:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.478536:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.478544:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24240128->24244223] (req 24240128->24244223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.478640:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508443264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24985us (25690us total) trans 4294973238 rc 0/0 00000100:00100000:0.0:1789760778.478650:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5950 00000100:00100000:0.0:1789760778.478660:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757529027441 time 757558341290 state 0x2 ref 0 00000100:00100000:2.0:1789760778.482754:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760778.482896:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508443392 00000100:00100000:3.0:1789760778.482907:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508443392 00000100:00100000:3.0:1789760778.482966:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757529027441 time 757558341290 state 0x0 ref 1 00000100:00100000:3.0:1789760778.482982:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757558601706 time 757558341290 state 0x1 ref 1 00000100:00100000:3.0:1789760778.483032:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757558601706 time 757587733668 state 0x0 ref 1 00000100:00100000:3.0:1789760778.483047:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508443392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760778.483096:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24244224, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0xc007 00010000:00010000:3.0:1789760778.483105:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe568e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.483119:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760778.483123:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.483129:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe568e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.483142:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe568e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760778.523684:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24244224 -> 24248320 00002000:00010000:0.0:1789760778.523702:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760778 -> 1789760779 00002000:00010000:0.0:1789760778.523705:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760778 -> 1789760779 00010000:00010000:0.0:1789760778.523724:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe568e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.523737:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe568e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.523746:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe568e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.523760:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760778.523764:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe568e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.523773:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe568e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.523792:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe568e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.523811:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe568e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24244224->24248319] (req 24244224->24248319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.523910:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508443392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40860us (41162us total) trans 4294973239 rc 0/0 00000100:00100000:0.0:1789760778.523921:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5951 00000100:00100000:0.0:1789760778.523932:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757558601706 time 757587733668 state 0x2 ref 0 00000001:00080000:1.0:1789760778.524169:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973239 is committed 00000100:00100000:3.0:1789760778.527429:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760778.530574:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508443520 00000100:00100000:1.0:1789760778.530591:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508443520 00000100:00100000:1.0:1789760778.530658:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757558601706 time 757587733668 state 0x0 ref 1 00000100:00100000:1.0:1789760778.530675:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757587994084 time 757587733668 state 0x1 ref 1 00000100:00100000:1.0:1789760778.530717:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757587994084 time 757635427610 state 0x0 ref 1 00000100:00100000:1.0:1789760778.530730:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508443520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760778.530781:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24248320, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:1.0:1789760778.530790:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe568ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.530824:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760778.530829:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.530940:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe568ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.530962:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe568ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.580583:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973240 is committed 00002000:00010000:3.0:1789760778.580672:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24248320 -> 24252416 00010000:00010000:3.0:1789760778.580688:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe568ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.580702:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe568ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.580717:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe568ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.580725:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760778.580729:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe568ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.580739:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe568ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.580759:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe568ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.580767:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe568ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24248320->24252415] (req 24248320->24252415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.580971:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508443520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50237us (53545us total) trans 4294973240 rc 0/0 00000100:00100000:3.0:1789760778.580982:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5952 00000100:00100000:3.0:1789760778.580992:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757587994084 time 757635427610 state 0x2 ref 0 00000100:00100000:3.0:1789760778.589674:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760778.589780:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508443648 00000100:00100000:0.0:1789760778.589790:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508443648 00000100:00100000:0.0:1789760778.589855:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757587994084 time 757635427610 state 0x0 ref 1 00000100:00100000:0.0:1789760778.589871:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757635688026 time 757635427610 state 0x1 ref 1 00000100:00100000:0.0:1789760778.589905:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757635688026 time 757694615370 state 0x0 ref 1 00000100:00100000:0.0:1789760778.589921:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508443648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760778.589972:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24252416, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:0.0:1789760778.589981:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.589998:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760778.590002:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.590009:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.590023:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.621781:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973241 is committed 00002000:00010000:3.0:1789760778.621967:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24252416 -> 24256512 00010000:00010000:3.0:1789760778.621984:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.622001:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.622011:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.622018:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760778.622024:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.622033:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.622053:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.622062:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe568f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24252416->24256511] (req 24252416->24256511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.622446:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508443648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32225us (32481us total) trans 4294973241 rc 0/0 00000100:00100000:3.0:1789760778.622458:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5953 00000100:00100000:3.0:1789760778.622467:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757635688026 time 757694615370 state 0x2 ref 0 00000100:00100000:2.0:1789760778.630998:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760778.631095:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508443776 00000100:00100000:0.0:1789760778.631104:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508443776 00000100:00100000:0.0:1789760778.631168:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757635688026 time 757694615370 state 0x0 ref 1 00000100:00100000:0.0:1789760778.631182:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757694875786 time 757694615370 state 0x1 ref 1 00000100:00100000:0.0:1789760778.631210:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757694875786 time 757735920953 state 0x0 ref 1 00000100:00100000:0.0:1789760778.631222:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508443776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760778.631269:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24256512, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:0.0:1789760778.631279:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.631327:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760778.631331:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.631338:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.631352:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.659946:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973242 is committed 00002000:00010000:1.0:1789760778.660477:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24256512 -> 24260608 00010000:00010000:1.0:1789760778.660492:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.660504:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.660512:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.660519:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760778.660524:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.660533:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.660554:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.660562:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe568fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24256512->24260607] (req 24256512->24260607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.660659:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508443776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29434us (29666us total) trans 4294973242 rc 0/0 00000100:00100000:1.0:1789760778.660669:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5954 00000100:00100000:1.0:1789760778.660679:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757694875786 time 757735920953 state 0x2 ref 0 00000100:00100000:3.0:1789760778.665172:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760778.665952:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508443904 00000100:00100000:1.0:1789760778.665964:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508443904 00000100:00100000:1.0:1789760778.666022:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757694875786 time 757735920953 state 0x0 ref 1 00000100:00100000:1.0:1789760778.666036:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757736181369 time 757735920953 state 0x1 ref 1 00000100:00100000:1.0:1789760778.666068:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757736181369 time 757770778570 state 0x0 ref 1 00000100:00100000:1.0:1789760778.666081:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508443904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760778.666131:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24260608, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:1.0:1789760778.666141:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56904 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.666166:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760778.666170:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.666177:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56904 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.666191:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56904 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760778.700890:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24260608 -> 24264704 00010000:00010000:3.0:1789760778.700912:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56904 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.700925:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56904 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.700934:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56904 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.700941:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760778.700946:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56904 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.700955:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56904 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.700977:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56904 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.700984:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56904 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24260608->24264703] (req 24260608->24264703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.701078:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508443904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34994us (35911us total) trans 4294973243 rc 0/0 00000100:00100000:3.0:1789760778.701090:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5955 00000100:00100000:3.0:1789760778.701100:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757736181369 time 757770778570 state 0x2 ref 0 00000001:00080000:0.0:1789760778.701632:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973243 is committed 00000100:00100000:3.0:1789760778.703694:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760778.704044:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508444032 00000100:00100000:0.0:1789760778.704054:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508444032 00000100:00100000:0.0:1789760778.704109:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757736181369 time 757770778570 state 0x0 ref 1 00000100:00100000:0.0:1789760778.704124:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757771038986 time 757770778570 state 0x1 ref 1 00000100:00100000:3.0:1789760778.704228:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757771038986 time 757808936250 state 0x0 ref 1 00000100:00100000:3.0:1789760778.704245:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d40850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508444032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760778.704534:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24264704, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:3.0:1789760778.704545:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5690b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.704561:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760778.704565:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.704572:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5690b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.704585:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5690b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.732423:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973244 is committed 00002000:00010000:3.0:1789760778.732592:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24264704 -> 24268800 00010000:00010000:3.0:1789760778.732607:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5690b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.732620:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5690b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.732629:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5690b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.732644:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760778.732649:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5690b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.732659:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5690b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.732681:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5690b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.732689:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5690b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24264704->24268799] (req 24264704->24268799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.732805:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d40850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508444032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28551us (29107us total) trans 4294973244 rc 0/0 00000100:00100000:3.0:1789760778.732816:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5956 00000100:00100000:3.0:1789760778.732826:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757771038986 time 757808936250 state 0x2 ref 0 00000100:00100000:2.0:1789760778.740899:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760778.740969:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508444160 00000100:00100000:1.0:1789760778.740979:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508444160 00000100:00100000:1.0:1789760778.741031:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757771038986 time 757808936250 state 0x0 ref 1 00000100:00100000:1.0:1789760778.741046:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757809196666 time 757808936250 state 0x1 ref 1 00000100:00100000:1.0:1789760778.741078:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757809196666 time 757845787911 state 0x0 ref 1 00000100:00100000:1.0:1789760778.741091:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508444160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760778.741144:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24268800, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:1.0:1789760778.741154:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56912 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.741169:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:1.0:1789760778.741174:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.741181:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56912 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.741194:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56912 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760778.785414:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24268800 -> 24272896 00010000:00010000:0.0:1789760778.785436:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56912 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.785449:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56912 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.785459:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56912 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.785466:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760778.785472:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56912 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.785481:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56912 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.785502:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56912 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.785509:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56912 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24268800->24272895] (req 24268800->24272895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.785769:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508444160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44675us (44874us total) trans 4294973245 rc 0/0 00000100:00100000:0.0:1789760778.785780:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5957 00000100:00100000:0.0:1789760778.785791:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757809196666 time 757845787911 state 0x2 ref 0 00000001:00080000:2.0:1789760778.786207:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973245 is committed 00000100:00100000:0.0:1789760778.789944:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760778.790030:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508444288 00000100:00100000:3.0:1789760778.790041:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508444288 00000100:00100000:3.0:1789760778.790110:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757809196666 time 757845787911 state 0x0 ref 1 00000100:00100000:3.0:1789760778.790126:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757846048327 time 757845787911 state 0x1 ref 1 00000100:00100000:3.0:1789760778.790152:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757846048327 time 757894862276 state 0x0 ref 1 00000100:00100000:3.0:1789760778.790169:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508444288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760778.790218:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24272896, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:3.0:1789760778.790227:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56919 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.790242:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:3.0:1789760778.790247:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.790253:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56919 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.790265:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56919 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760778.823884:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24272896 -> 24276992 00010000:00010000:3.0:1789760778.823905:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56919 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.823918:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56919 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.823926:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56919 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.823933:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760778.823938:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56919 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.823948:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56919 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.823967:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56919 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.823974:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56919 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24272896->24276991] (req 24272896->24276991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.824105:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508444288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33934us (34165us total) trans 4294973246 rc 0/0 00000100:00100000:3.0:1789760778.824116:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5958 00000100:00100000:3.0:1789760778.824125:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757846048327 time 757894862276 state 0x2 ref 0 00000001:00080000:2.0:1789760778.824269:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973246 is committed 00000100:00100000:3.0:1789760778.829485:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760778.829550:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508444416 00000100:00100000:2.0:1789760778.829560:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508444416 00000100:00100000:2.0:1789760778.829619:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757846048327 time 757894862276 state 0x0 ref 1 00000100:00100000:2.0:1789760778.829646:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757895122692 time 757894862276 state 0x1 ref 1 00000100:00100000:2.0:1789760778.829745:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757895122692 time 757934447663 state 0x0 ref 1 00000100:00100000:2.0:1789760778.829761:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508444416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760778.829822:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24276992, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:2.0:1789760778.829832:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56920 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.829854:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:2.0:1789760778.829859:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.829866:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56920 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.829879:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56920 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760778.863858:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24276992 -> 24281088 00010000:00010000:1.0:1789760778.863876:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56920 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.863894:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56920 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.863903:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56920 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.863910:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760778.863915:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56920 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.863923:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56920 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.863945:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56920 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.863953:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56920 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24276992->24281087] (req 24276992->24281087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.864047:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508444416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34285us (34569us total) trans 4294973247 rc 0/0 00000100:00100000:1.0:1789760778.864057:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5959 00000100:00100000:1.0:1789760778.864067:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757895122692 time 757934447663 state 0x2 ref 0 00000001:00080000:2.0:1789760778.865022:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973247 is committed 00000100:00100000:1.0:1789760778.868947:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760778.869031:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508444544 00000100:00100000:0.0:1789760778.869042:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508444544 00000100:00100000:0.0:1789760778.869095:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757895122692 time 757934447663 state 0x0 ref 1 00000100:00100000:0.0:1789760778.869111:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757934708079 time 757934447663 state 0x1 ref 1 00000100:00100000:0.0:1789760778.869142:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757934708079 time 757973852376 state 0x0 ref 1 00000100:00100000:0.0:1789760778.869163:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508444544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760778.869212:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24281088, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:0.0:1789760778.869222:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56927 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.869237:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=757s v=(0 0) 00010000:00010000:0.0:1789760778.869242:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.869248:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56927 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.869263:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56927 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.903685:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973248 is committed 00002000:00010000:3.0:1789760778.903972:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24281088 -> 24285184 00010000:00010000:3.0:1789760778.903988:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56927 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.904001:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56927 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.904010:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56927 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.904022:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760778.904026:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56927 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.904036:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56927 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.904058:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56927 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.904066:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56927 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24281088->24285183] (req 24281088->24285183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.904183:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508444544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35017us (35241us total) trans 4294973248 rc 0/0 00000100:00100000:3.0:1789760778.904194:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5960 00000100:00100000:3.0:1789760778.904214:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757934708079 time 757973852376 state 0x2 ref 0 00000100:00100000:1.0:1789760778.911350:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760778.911759:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508444672 00000100:00100000:3.0:1789760778.911771:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508444672 00000100:00100000:3.0:1789760778.911840:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757934708079 time 757973852376 state 0x0 ref 1 00000100:00100000:3.0:1789760778.911856:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757974112792 time 757973852376 state 0x1 ref 1 00000100:00100000:3.0:1789760778.911925:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757974112792 time 758016635337 state 0x0 ref 1 00000100:00100000:3.0:1789760778.911942:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508444672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760778.911990:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24285184, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:3.0:1789760778.912000:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5692e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.912016:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760778.912021:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.912027:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5692e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.912042:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5692e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760778.976151:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24285184 -> 24289280 00010000:00010000:3.0:1789760778.976174:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5692e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.976188:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5692e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.976198:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5692e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.976205:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760778.976210:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5692e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.976221:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5692e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.976242:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5692e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.976250:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5692e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24285184->24289279] (req 24285184->24289279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.976381:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508444672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 64436us (65036us total) trans 4294973249 rc 0/0 00000100:00100000:3.0:1789760778.976393:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5961 00000100:00100000:3.0:1789760778.976403:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757974112792 time 758016635337 state 0x2 ref 0 00000001:00080000:0.0:1789760778.976940:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973249 is committed 00000100:00100000:2.0:1789760778.980598:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760778.980680:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508444800 00000100:00100000:1.0:1789760778.980692:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508444800 00000100:00100000:1.0:1789760778.980754:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 757974112792 time 758016635337 state 0x0 ref 1 00000100:00100000:1.0:1789760778.980770:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758016895753 time 758016635337 state 0x1 ref 1 00000100:00100000:1.0:1789760778.980805:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758016895753 time 758085514587 state 0x0 ref 1 00000100:00100000:1.0:1789760778.980817:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508444800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760778.980866:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24289280, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:1.0:1789760778.980875:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56935 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.980891:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760778.980896:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.980904:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56935 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.980921:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56935 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.008400:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760779.012483:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24289280 -> 24293376 00010000:00010000:1.0:1789760779.012505:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56935 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.012519:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56935 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.012530:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56935 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.012537:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760779.012542:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56935 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.012551:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56935 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.012571:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56935 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.012577:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56935 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24289280->24293375] (req 24289280->24293375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.012660:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973250 is committed 00000100:00100000:1.0:1789760779.012713:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508444800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31892us (32120us total) trans 4294973250 rc 0/0 00000100:00100000:1.0:1789760779.012721:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5962 00000100:00100000:1.0:1789760779.012730:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758016895753 time 758085514587 state 0x2 ref 0 00000100:00100000:3.0:1789760779.017218:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760779.017484:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508444928 00000100:00100000:0.0:1789760779.017495:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508444928 00000100:00080000:1.0:1789760779.017538:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760779.017571:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758016895753 time 758085514587 state 0x0 ref 1 00000100:00100000:0.0:1789760779.017587:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758085775003 time 758085514587 state 0x1 ref 1 00000100:00100000:0.0:1789760779.017617:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758085775003 time 758122327616 state 0x0 ref 1 00000100:00100000:0.0:1789760779.017629:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508444928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760779.017681:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24293376, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:0.0:1789760779.017690:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5693c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.017711:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760779.017716:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.017722:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5693c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.017736:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5693c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760779.056317:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24293376 -> 24297472 00010000:00010000:3.0:1789760779.056338:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5693c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.056353:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5693c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.056362:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5693c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.056374:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760779.056380:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5693c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.056390:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5693c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.056412:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5693c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.056420:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe5693c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24293376->24297471] (req 24293376->24297471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.056518:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508444928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38885us (39304us total) trans 4294973251 rc 0/0 00000100:00100000:3.0:1789760779.056528:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5963 00000100:00100000:3.0:1789760779.056538:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758085775003 time 758122327616 state 0x2 ref 0 00000001:00080000:1.0:1789760779.056846:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973251 is committed 00000100:00100000:3.0:1789760779.061404:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760779.061536:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508445056 00000100:00100000:1.0:1789760779.061546:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508445056 00000100:00100000:1.0:1789760779.061598:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758085775003 time 758122327616 state 0x0 ref 1 00000100:00100000:1.0:1789760779.061613:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758122588032 time 758122327616 state 0x1 ref 1 00000100:00100000:1.0:1789760779.061642:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758122588032 time 758166352983 state 0x0 ref 1 00000100:00100000:1.0:1789760779.061655:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508445056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760779.061702:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24297472, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:1.0:1789760779.061711:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56943 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.061739:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.061744:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.061750:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56943 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.061764:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56943 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.091009:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973252 is committed 00002000:00010000:1.0:1789760779.091421:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24297472 -> 24301568 00010000:00010000:1.0:1789760779.091437:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56943 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.091450:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56943 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.091461:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56943 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.091473:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760779.091478:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56943 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.091487:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56943 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.091507:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56943 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.091516:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56943 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24297472->24301567] (req 24297472->24301567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.091608:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508445056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29951us (30209us total) trans 4294973252 rc 0/0 00000100:00100000:1.0:1789760779.091618:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5964 00000100:00100000:1.0:1789760779.091627:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758122588032 time 758166352983 state 0x2 ref 0 00000100:00100000:2.0:1789760779.106461:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760779.106571:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508445184 00000100:00100000:1.0:1789760779.106581:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508445184 00000100:00100000:1.0:1789760779.106639:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758122588032 time 758166352983 state 0x0 ref 1 00000100:00100000:1.0:1789760779.106655:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758166613399 time 758166352983 state 0x1 ref 1 00000100:00100000:1.0:1789760779.106693:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758166613399 time 758211402950 state 0x0 ref 1 00000100:00100000:1.0:1789760779.106719:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508445184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760779.106771:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24301568, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:1.0:1789760779.106780:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5694a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.106805:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.106810:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.106816:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5694a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.106830:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5694a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.142899:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973253 is committed 00002000:00010000:0.0:1789760779.143004:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24301568 -> 24305664 00010000:00010000:0.0:1789760779.143019:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5694a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.143032:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5694a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.143041:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5694a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.143049:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760779.143053:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5694a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.143062:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5694a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.143083:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5694a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.143091:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5694a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24301568->24305663] (req 24301568->24305663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.143277:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508445184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36565us (36822us total) trans 4294973253 rc 0/0 00000100:00100000:0.0:1789760779.143314:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5965 00000100:00100000:0.0:1789760779.143324:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758166613399 time 758211402950 state 0x2 ref 0 00000100:00100000:3.0:1789760779.147097:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760779.147198:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508445312 00000100:00100000:0.0:1789760779.147207:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508445312 00000100:00100000:0.0:1789760779.147263:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758166613399 time 758211402950 state 0x0 ref 1 00000100:00100000:0.0:1789760779.147276:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758211663366 time 758211402950 state 0x1 ref 1 00000100:00100000:0.0:1789760779.147317:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758211663366 time 758252027887 state 0x0 ref 1 00000100:00100000:0.0:1789760779.147328:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508445312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760779.147375:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24305664, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:0.0:1789760779.147384:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56951 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.147398:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760779.147403:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.147409:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56951 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.147426:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56951 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.179631:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973254 is committed 00002000:00010000:0.0:1789760779.179660:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24305664 -> 24309760 00010000:00010000:0.0:1789760779.179675:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56951 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.179693:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56951 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.179702:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56951 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.179710:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760779.179716:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56951 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.179729:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56951 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.179749:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56951 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.179757:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56951 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24305664->24309759] (req 24305664->24309759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.179889:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508445312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32557us (32796us total) trans 4294973254 rc 0/0 00000100:00100000:0.0:1789760779.179900:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5966 00000100:00100000:0.0:1789760779.179910:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758211663366 time 758252027887 state 0x2 ref 0 00000100:00100000:3.0:1789760779.185087:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760779.185164:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508445440 00000100:00100000:1.0:1789760779.185181:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508445440 00000100:00100000:1.0:1789760779.185235:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758211663366 time 758252027887 state 0x0 ref 1 00000100:00100000:1.0:1789760779.185250:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758252288303 time 758252027887 state 0x1 ref 1 00000100:00100000:1.0:1789760779.185414:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758252288303 time 758290123313 state 0x0 ref 1 00000100:00100000:1.0:1789760779.185428:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508445440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760779.185476:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24309760, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:1.0:1789760779.185485:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56958 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.185505:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.185510:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.185554:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56958 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.185580:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56958 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760779.215600:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24309760 -> 24313856 00010000:00010000:1.0:1789760779.215621:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56958 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.215634:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56958 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.215643:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56958 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.215651:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760779.215656:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56958 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.215665:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56958 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.215685:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56958 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.215693:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56958 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24309760->24313855] (req 24309760->24313855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.215806:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508445440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30367us (30716us total) trans 4294973255 rc 0/0 00000100:00100000:1.0:1789760779.215817:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5967 00000100:00100000:1.0:1789760779.215827:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758252288303 time 758290123313 state 0x2 ref 0 00000001:00080000:3.0:1789760779.216344:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973255 is committed 00000100:00100000:2.0:1789760779.218597:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760779.218677:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508445568 00000100:00100000:1.0:1789760779.218687:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508445568 00000100:00100000:1.0:1789760779.218757:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758252288303 time 758290123313 state 0x0 ref 1 00000100:00100000:1.0:1789760779.218773:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758290383729 time 758290123313 state 0x1 ref 1 00000100:00100000:1.0:1789760779.218810:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758290383729 time 758323520707 state 0x0 ref 1 00000100:00100000:1.0:1789760779.218822:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508445568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760779.218870:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24313856, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:1.0:1789760779.218877:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5695f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.218892:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.218897:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.218904:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5695f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.218924:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5695f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.251081:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973256 is committed 00002000:00010000:3.0:1789760779.251208:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24313856 -> 24317952 00010000:00010000:3.0:1789760779.251224:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5695f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.251238:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5695f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.251247:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5695f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.251254:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760779.251259:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5695f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.251268:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5695f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.251317:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5695f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.251325:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5695f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24313856->24317951] (req 24313856->24317951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.251453:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508445568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32628us (32860us total) trans 4294973256 rc 0/0 00000100:00100000:3.0:1789760779.251464:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5968 00000100:00100000:3.0:1789760779.251474:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758290383729 time 758323520707 state 0x2 ref 0 00000100:00100000:2.0:1789760779.256915:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760779.256981:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508445696 00000100:00100000:0.0:1789760779.256991:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508445696 00000100:00100000:0.0:1789760779.257042:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758290383729 time 758323520707 state 0x0 ref 1 00000100:00100000:0.0:1789760779.257056:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758323781123 time 758323520707 state 0x1 ref 1 00000100:00100000:0.0:1789760779.257088:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758323781123 time 758361798356 state 0x0 ref 1 00000100:00100000:0.0:1789760779.257100:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508445696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760779.257149:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24317952, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:0.0:1789760779.257157:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56966 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.257172:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760779.257176:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.257182:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56966 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.257196:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56966 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.295379:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973257 is committed 00002000:00010000:0.0:1789760779.295483:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24317952 -> 24322048 00010000:00010000:0.0:1789760779.295499:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56966 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.295551:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56966 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.295562:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56966 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.295568:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760779.295573:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56966 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.295582:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56966 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.295602:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56966 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.295608:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56966 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24317952->24322047] (req 24317952->24322047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.295699:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508445696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38595us (38788us total) trans 4294973257 rc 0/0 00000100:00100000:0.0:1789760779.295712:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5969 00000100:00100000:0.0:1789760779.295720:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758323781123 time 758361798356 state 0x2 ref 0 00000100:00100000:1.0:1789760779.299520:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760779.299636:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508445824 00000100:00100000:2.0:1789760779.299645:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508445824 00000100:00100000:2.0:1789760779.299702:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758323781123 time 758361798356 state 0x0 ref 1 00000100:00100000:2.0:1789760779.299716:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758362058772 time 758361798356 state 0x1 ref 1 00000100:00100000:2.0:1789760779.299785:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758362058772 time 758404495061 state 0x0 ref 1 00000100:00100000:2.0:1789760779.299808:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508445824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760779.299858:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24322048, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:2.0:1789760779.299866:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5696d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.299881:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:2.0:1789760779.299886:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.299892:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5696d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.299906:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5696d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.342144:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973258 is committed 00002000:00010000:2.0:1789760779.342150:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24322048 -> 24326144 00010000:00010000:2.0:1789760779.342165:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5696d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.342179:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5696d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.342188:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5696d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.342196:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760779.342201:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5696d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.342210:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5696d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.342236:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5696d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.342244:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5696d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24322048->24326143] (req 24322048->24326143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.342463:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508445824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42653us (42947us total) trans 4294973258 rc 0/0 00000100:00100000:2.0:1789760779.342474:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5970 00000100:00100000:2.0:1789760779.342484:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758362058772 time 758404495061 state 0x2 ref 0 00000100:00100000:2.0:1789760779.345498:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760779.345577:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508445952 00000100:00100000:3.0:1789760779.345589:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508445952 00000100:00100000:3.0:1789760779.345643:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758362058772 time 758404495061 state 0x0 ref 1 00000100:00100000:3.0:1789760779.345659:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758404755477 time 758404495061 state 0x1 ref 1 00000100:00100000:3.0:1789760779.345687:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758404755477 time 758450398350 state 0x0 ref 1 00000100:00100000:3.0:1789760779.345699:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508445952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760779.345860:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24326144, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:3.0:1789760779.345870:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56974 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.345885:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760779.345889:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.345896:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56974 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.345910:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56974 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760779.372714:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24326144 -> 24330240 00010000:00010000:1.0:1789760779.372732:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56974 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.372745:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56974 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.372754:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56974 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.372761:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760779.372766:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56974 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.372775:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56974 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.372803:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56974 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.372810:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56974 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24326144->24330239] (req 24326144->24330239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.372921:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508445952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27218us (27426us total) trans 4294973259 rc 0/0 00000100:00100000:1.0:1789760779.372931:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5971 00000100:00100000:1.0:1789760779.372940:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758404755477 time 758450398350 state 0x2 ref 0 00000001:00080000:0.0:1789760779.373392:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973259 is committed 00000100:00100000:1.0:1789760779.386114:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760779.386225:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508446080 00000100:00100000:0.0:1789760779.386235:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508446080 00000100:00100000:0.0:1789760779.386420:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758404755477 time 758450398350 state 0x0 ref 1 00000100:00100000:0.0:1789760779.386437:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758450658766 time 758450398350 state 0x1 ref 1 00000100:00100000:3.0:1789760779.386529:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758450658766 time 758491236949 state 0x0 ref 1 00000100:00100000:3.0:1789760779.386544:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508446080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760779.386597:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24330240, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:3.0:1789760779.386606:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5697b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.386626:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760779.386630:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.386636:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5697b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.386649:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5697b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760779.417721:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24330240 -> 24334336 00010000:00010000:3.0:1789760779.417740:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5697b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.417753:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5697b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.417762:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5697b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.417769:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760779.417773:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5697b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.417781:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5697b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.417809:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5697b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.417817:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5697b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24330240->24334335] (req 24330240->24334335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.417909:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508446080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31364us (31800us total) trans 4294973260 rc 0/0 00000100:00100000:3.0:1789760779.417920:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5972 00000100:00100000:3.0:1789760779.417930:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758450658766 time 758491236949 state 0x2 ref 0 00000001:00080000:0.0:1789760779.418433:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973260 is committed 00000100:00100000:2.0:1789760779.434391:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760779.434553:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508446208 00000100:00100000:1.0:1789760779.434563:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508446208 00000100:00100000:1.0:1789760779.434619:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758450658766 time 758491236949 state 0x0 ref 1 00000100:00100000:1.0:1789760779.434635:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758491497365 time 758491236949 state 0x1 ref 1 00000100:00100000:1.0:1789760779.434656:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758491497365 time 758539367447 state 0x0 ref 1 00000100:00100000:1.0:1789760779.434668:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508446208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760779.434716:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24334336, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:1.0:1789760779.434725:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56982 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.434740:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.434744:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.434750:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56982 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.434764:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56982 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.467604:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973261 is committed 00002000:00010000:1.0:1789760779.467702:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24334336 -> 24338432 00010000:00010000:1.0:1789760779.467717:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56982 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.467730:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56982 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.467740:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56982 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.467747:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760779.467751:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56982 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.467760:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56982 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.467779:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56982 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.467787:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56982 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24334336->24338431] (req 24334336->24338431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.467956:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508446208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33285us (33568us total) trans 4294973261 rc 0/0 00000100:00100000:1.0:1789760779.467970:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5973 00000100:00100000:1.0:1789760779.467979:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758491497365 time 758539367447 state 0x2 ref 0 00000100:00100000:2.0:1789760779.471691:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760779.472641:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508446336 00000100:00100000:1.0:1789760779.472651:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508446336 00000100:00100000:1.0:1789760779.472709:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758491497365 time 758539367447 state 0x0 ref 1 00000100:00100000:1.0:1789760779.472722:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758539627863 time 758539367447 state 0x1 ref 1 00000100:00100000:1.0:1789760779.472749:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758539627863 time 758577459607 state 0x0 ref 1 00000100:00100000:1.0:1789760779.472761:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508446336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760779.472822:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24338432, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0xc007 00010000:00010000:1.0:1789760779.472831:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56989 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.472946:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.472951:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.472958:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56989 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.472973:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56989 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.503166:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24338432 -> 24342528 00002000:00010000:2.0:1789760779.503175:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760779 -> 1789760780 00002000:00010000:2.0:1789760779.503177:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760779 -> 1789760780 00010000:00010000:2.0:1789760779.503190:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56989 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.503202:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56989 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.503211:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56989 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.503219:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760779.503224:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56989 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.503233:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56989 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.503252:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56989 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.503260:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56989 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24338432->24342527] (req 24338432->24342527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.503384:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508446336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30620us (31698us total) trans 4294973262 rc 0/0 00000100:00100000:2.0:1789760779.503394:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5974 00000100:00100000:2.0:1789760779.503404:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758539627863 time 758577459607 state 0x2 ref 0 00000001:00080000:0.0:1789760779.503566:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973262 is committed 00000100:00100000:2.0:1789760779.507560:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760779.507673:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508446464 00000100:00100000:3.0:1789760779.507682:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508446464 00000100:00100000:3.0:1789760779.507881:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758539627863 time 758577459607 state 0x0 ref 1 00000100:00100000:3.0:1789760779.507897:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758577720023 time 758577459607 state 0x1 ref 1 00000100:00100000:3.0:1789760779.507967:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758577720023 time 758612676849 state 0x0 ref 1 00000100:00100000:3.0:1789760779.507982:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508446464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760779.508028:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24342528, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:3.0:1789760779.508037:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56990 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.508052:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760779.508057:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.508063:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56990 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.508083:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56990 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.539241:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973263 is committed 00002000:00010000:0.0:1789760779.539279:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24342528 -> 24346624 00010000:00010000:0.0:1789760779.539311:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56990 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.539325:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56990 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.539334:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56990 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.539340:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760779.539344:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56990 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.539357:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56990 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.539375:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56990 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.539383:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56990 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24342528->24346623] (req 24342528->24346623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.539493:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508446464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31510us (31937us total) trans 4294973263 rc 0/0 00000100:00100000:0.0:1789760779.539504:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5975 00000100:00100000:0.0:1789760779.539624:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758577720023 time 758612676849 state 0x2 ref 0 00000100:00100000:2.0:1789760779.552596:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760779.552685:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508446592 00000100:00100000:1.0:1789760779.552696:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508446592 00000100:00100000:1.0:1789760779.552751:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758577720023 time 758612676849 state 0x0 ref 1 00000100:00100000:1.0:1789760779.552766:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758612937265 time 758612676849 state 0x1 ref 1 00000100:00100000:1.0:1789760779.552894:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758612937265 time 758657538359 state 0x0 ref 1 00000100:00100000:1.0:1789760779.552910:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508446592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760779.552960:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24346624, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:1.0:1789760779.552970:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56997 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.552986:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.552990:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.552995:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56997 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.553006:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56997 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.590528:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973264 is committed 00002000:00010000:1.0:1789760779.590681:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24346624 -> 24350720 00010000:00010000:1.0:1789760779.590697:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56997 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.590710:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56997 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.590721:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56997 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.590728:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760779.590733:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56997 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.590742:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56997 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.590761:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56997 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.590769:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56997 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24346624->24350719] (req 24346624->24350719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.590876:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508446592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37964us (38285us total) trans 4294973264 rc 0/0 00000100:00100000:1.0:1789760779.590886:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5976 00000100:00100000:1.0:1789760779.590906:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758612937265 time 758657538359 state 0x2 ref 0 00000100:00100000:2.0:1789760779.595781:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760779.596016:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508446720 00000100:00100000:1.0:1789760779.596026:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508446720 00000100:00100000:1.0:1789760779.596116:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758612937265 time 758657538359 state 0x0 ref 1 00000100:00100000:1.0:1789760779.596131:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758657798775 time 758657538359 state 0x1 ref 1 00000100:00100000:1.0:1789760779.596168:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758657798775 time 758700878277 state 0x0 ref 1 00000100:00100000:1.0:1789760779.596184:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508446720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760779.596234:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24350720, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:1.0:1789760779.596243:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5699e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.596259:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.596263:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.596269:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5699e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.596324:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5699e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.635099:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973265 is committed 00002000:00010000:2.0:1789760779.635391:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24350720 -> 24354816 00010000:00010000:2.0:1789760779.635406:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5699e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.635419:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5699e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.635430:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5699e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.635438:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760779.635443:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5699e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.635452:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5699e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.635471:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5699e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.635478:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5699e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24350720->24354815] (req 24350720->24354815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.636049:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508446720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39860us (40274us total) trans 4294973265 rc 0/0 00000100:00100000:3.0:1789760779.636061:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5977 00000100:00100000:3.0:1789760779.636070:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758657798775 time 758700878277 state 0x2 ref 0 00000100:00100000:2.0:1789760779.641041:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760779.641195:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508446848 00000100:00100000:0.0:1789760779.641206:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508446848 00000100:00100000:0.0:1789760779.641264:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758657798775 time 758700878277 state 0x0 ref 1 00000100:00100000:0.0:1789760779.641408:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758701138693 time 758700878277 state 0x1 ref 1 00000100:00100000:0.0:1789760779.641440:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758701138693 time 758746150382 state 0x0 ref 1 00000100:00100000:0.0:1789760779.641454:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508446848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760779.641512:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24354816, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:0.0:1789760779.641528:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.641544:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760779.641548:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.641555:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.641569:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760779.679466:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24354816 -> 24358912 00010000:00010000:0.0:1789760779.679485:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.679498:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.679508:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.679515:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760779.679519:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.679527:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.679546:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.679555:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24354816->24358911] (req 24354816->24358911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.679673:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973266 is committed 00000100:00100000:0.0:1789760779.679684:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508446848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38228us (38703us total) trans 4294973266 rc 0/0 00000100:00100000:0.0:1789760779.679694:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5978 00000100:00100000:0.0:1789760779.679704:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758701138693 time 758746150382 state 0x2 ref 0 00000100:00100000:2.0:1789760779.683272:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760779.683457:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508446976 00000100:00100000:1.0:1789760779.683467:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508446976 00000100:00100000:1.0:1789760779.683533:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758701138693 time 758746150382 state 0x0 ref 1 00000100:00100000:1.0:1789760779.683549:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758746410798 time 758746150382 state 0x1 ref 1 00000100:00100000:1.0:1789760779.683609:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758746410798 time 758788319593 state 0x0 ref 1 00000100:00100000:1.0:1789760779.683622:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508446976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760779.683668:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24358912, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:1.0:1789760779.683675:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe569ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.683692:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.683695:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.683699:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe569ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.683708:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe569ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760779.708783:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24358912 -> 24363008 00010000:00010000:2.0:1789760779.708812:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe569ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.708826:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe569ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.708836:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe569ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.708843:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760779.708848:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe569ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.708857:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe569ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.708876:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe569ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.708884:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe569ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24358912->24363007] (req 24358912->24363007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.709035:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973267 is committed 00000100:00100000:2.0:1789760779.709086:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508446976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25461us (25819us total) trans 4294973267 rc 0/0 00000100:00100000:2.0:1789760779.709097:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5979 00000100:00100000:2.0:1789760779.709107:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758746410798 time 758788319593 state 0x2 ref 0 00000100:00100000:2.0:1789760779.713707:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760779.713810:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508447104 00000100:00100000:3.0:1789760779.713820:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508447104 00000100:00100000:3.0:1789760779.713875:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758746410798 time 758788319593 state 0x0 ref 1 00000100:00100000:3.0:1789760779.713890:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758788580009 time 758788319593 state 0x1 ref 1 00000100:00100000:3.0:1789760779.713946:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758788580009 time 758818656203 state 0x0 ref 1 00000100:00100000:3.0:1789760779.713959:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508447104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760779.714005:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24363008, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:3.0:1789760779.714014:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe569b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.714028:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:3.0:1789760779.714033:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.714040:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe569b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.714054:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe569b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.741370:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973268 is committed 00002000:00010000:3.0:1789760779.741521:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24363008 -> 24367104 00010000:00010000:3.0:1789760779.741538:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe569b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.741551:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe569b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.741561:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe569b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.741568:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760779.741573:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe569b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.741594:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe569b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.741614:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe569b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.741622:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe569b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24363008->24367103] (req 24363008->24367103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.741716:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508447104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27754us (28015us total) trans 4294973268 rc 0/0 00000100:00100000:3.0:1789760779.741726:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5980 00000100:00100000:3.0:1789760779.741735:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758788580009 time 758818656203 state 0x2 ref 0 00000100:00100000:1.0:1789760779.746841:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760779.747035:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508447232 00000100:00100000:0.0:1789760779.747047:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508447232 00000100:00100000:0.0:1789760779.747103:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758788580009 time 758818656203 state 0x0 ref 1 00000100:00100000:0.0:1789760779.747120:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758818916619 time 758818656203 state 0x1 ref 1 00000100:00100000:1.0:1789760779.747376:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758818916619 time 758852083212 state 0x0 ref 1 00000100:00100000:1.0:1789760779.747393:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508447232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760779.747447:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24367104, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:1.0:1789760779.747456:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe569ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.747473:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.747478:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.747484:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe569ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.747499:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe569ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.775567:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973269 is committed 00002000:00010000:0.0:1789760779.775661:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24367104 -> 24371200 00010000:00010000:0.0:1789760779.775676:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe569ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.775689:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe569ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.775699:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe569ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.775706:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760779.775711:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe569ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.775720:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe569ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.775739:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe569ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.775746:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe569ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24367104->24371199] (req 24367104->24371199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.775843:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508447232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28449us (29007us total) trans 4294973269 rc 0/0 00000100:00100000:0.0:1789760779.775854:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5981 00000100:00100000:0.0:1789760779.775863:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758818916619 time 758852083212 state 0x2 ref 0 00000100:00100000:2.0:1789760779.780468:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760779.780566:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508447360 00000100:00100000:1.0:1789760779.780578:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508447360 00000100:00100000:1.0:1789760779.780630:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758818916619 time 758852083212 state 0x0 ref 1 00000100:00100000:1.0:1789760779.780645:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758852343628 time 758852083212 state 0x1 ref 1 00000100:00100000:1.0:1789760779.780713:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758852343628 time 758885422486 state 0x0 ref 1 00000100:00100000:1.0:1789760779.780725:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508447360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760779.780772:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24371200, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:1.0:1789760779.780780:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe569c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.780803:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:1.0:1789760779.780807:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.780814:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe569c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.780827:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe569c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.803682:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973270 is committed 00002000:00010000:0.0:1789760779.803976:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24371200 -> 24375296 00010000:00010000:0.0:1789760779.803992:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe569c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.804005:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe569c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.804013:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe569c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.804026:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760779.804030:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe569c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.804038:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe569c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.804057:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe569c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.804064:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe569c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24371200->24375295] (req 24371200->24375295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.804209:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508447360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23481us (23747us total) trans 4294973270 rc 0/0 00000100:00100000:0.0:1789760779.804220:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5982 00000100:00100000:0.0:1789760779.804229:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758852343628 time 758885422486 state 0x2 ref 0 00000100:00100000:3.0:1789760779.807868:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760779.808000:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508447488 00000100:00100000:0.0:1789760779.808008:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508447488 00000100:00100000:0.0:1789760779.808053:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758852343628 time 758885422486 state 0x0 ref 1 00000100:00100000:0.0:1789760779.808064:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758885682902 time 758885422486 state 0x1 ref 1 00000100:00100000:0.0:1789760779.808116:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758885682902 time 758912825958 state 0x0 ref 1 00000100:00100000:0.0:1789760779.808133:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508447488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760779.808180:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24375296, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:0.0:1789760779.808189:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.808203:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760779.808208:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.808214:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.808226:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.829799:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973271 is committed 00002000:00010000:0.0:1789760779.829955:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24375296 -> 24379392 00010000:00010000:0.0:1789760779.829971:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.829983:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.829991:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.829999:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760779.830003:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.830012:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.830031:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.830038:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24375296->24379391] (req 24375296->24379391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.830129:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508447488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 21993us (22266us total) trans 4294973271 rc 0/0 00000100:00100000:0.0:1789760779.830148:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5983 00000100:00100000:0.0:1789760779.830158:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758885682902 time 758912825958 state 0x2 ref 0 00000100:00100000:1.0:1789760779.835843:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760779.836026:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508447616 00000100:00100000:0.0:1789760779.836035:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508447616 00000100:00100000:0.0:1789760779.836090:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758885682902 time 758912825958 state 0x0 ref 1 00000100:00100000:0.0:1789760779.836107:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758913086374 time 758912825958 state 0x1 ref 1 00000100:00100000:0.0:1789760779.836278:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758913086374 time 758940986774 state 0x0 ref 1 00000100:00100000:0.0:1789760779.836325:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508447616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760779.836387:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24379392, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:0.0:1789760779.836397:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.836413:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760779.836417:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.836423:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.836446:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.876055:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973272 is committed 00002000:00010000:0.0:1789760779.876249:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24379392 -> 24383488 00010000:00010000:0.0:1789760779.876265:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.876277:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.876358:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.876367:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760779.876372:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.876381:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.876402:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.876409:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24379392->24383487] (req 24379392->24383487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.876550:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508447616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40224us (40712us total) trans 4294973272 rc 0/0 00000100:00100000:0.0:1789760779.876561:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5984 00000100:00100000:0.0:1789760779.876571:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758913086374 time 758940986774 state 0x2 ref 0 00000100:00100000:1.0:1789760779.880003:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760779.880067:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508447744 00000100:00100000:0.0:1789760779.880078:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508447744 00000100:00100000:0.0:1789760779.880132:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758913086374 time 758940986774 state 0x0 ref 1 00000100:00100000:0.0:1789760779.880158:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758941247190 time 758940986774 state 0x1 ref 1 00000100:00100000:0.0:1789760779.880190:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758941247190 time 758984900753 state 0x0 ref 1 00000100:00100000:0.0:1789760779.880203:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508447744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760779.880254:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24383488, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:0.0:1789760779.880263:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.880278:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=758s v=(0 0) 00010000:00010000:0.0:1789760779.880282:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.880317:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.880336:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.919429:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973273 is committed 00002000:00010000:0.0:1789760779.921381:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24383488 -> 24387584 00010000:00010000:0.0:1789760779.921401:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.921414:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.921422:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.921429:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760779.921433:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.921442:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.921464:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.921473:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24383488->24387583] (req 24383488->24387583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.921629:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508447744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41424us (41630us total) trans 4294973273 rc 0/0 00000100:00100000:0.0:1789760779.921641:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5985 00000100:00100000:0.0:1789760779.921650:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758941247190 time 758984900753 state 0x2 ref 0 00000100:00100000:2.0:1789760779.925431:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760779.925498:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508447872 00000100:00100000:0.0:1789760779.925507:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508447872 00000100:00100000:0.0:1789760779.925553:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758941247190 time 758984900753 state 0x0 ref 1 00000100:00100000:0.0:1789760779.925567:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758985161169 time 758984900753 state 0x1 ref 1 00000100:00100000:0.0:1789760779.925594:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758985161169 time 759030304534 state 0x0 ref 1 00000100:00100000:0.0:1789760779.925604:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508447872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760779.925648:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24387584, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:0.0:1789760779.925657:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.925673:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760779.925678:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.925683:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.925697:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760779.968654:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24387584 -> 24391680 00010000:00010000:0.0:1789760779.968678:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.968691:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.968703:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.968714:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760779.968719:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.968728:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.968754:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.968766:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24387584->24391679] (req 24387584->24391679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.968883:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508447872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43274us (43455us total) trans 4294973274 rc 0/0 00000100:00100000:0.0:1789760779.968901:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5986 00000100:00100000:0.0:1789760779.968911:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758985161169 time 759030304534 state 0x2 ref 0 00000001:00080000:3.0:1789760779.969866:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973274 is committed 00000100:00100000:1.0:1789760779.973717:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760779.974665:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508448000 00000100:00100000:0.0:1789760779.974677:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508448000 00000100:00100000:0.0:1789760779.974736:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 758985161169 time 759030304534 state 0x0 ref 1 00000100:00100000:0.0:1789760779.974754:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759030564950 time 759030304534 state 0x1 ref 1 00000100:00100000:0.0:1789760779.974783:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759030564950 time 759079493626 state 0x0 ref 1 00000100:00100000:0.0:1789760779.974805:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508448000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760779.974860:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24391680, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:0.0:1789760779.974869:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.974885:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760779.974890:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.974896:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.974910:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.011499:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973275 is committed 00002000:00010000:0.0:1789760780.011678:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24391680 -> 24395776 00010000:00010000:0.0:1789760780.011694:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.011709:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.011718:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.011725:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760780.011729:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.011739:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.011759:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.011767:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24391680->24395775] (req 24391680->24395775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.011905:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508448000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37098us (38191us total) trans 4294973275 rc 0/0 00000100:00100000:0.0:1789760780.011915:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5987 00000100:00100000:0.0:1789760780.011925:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759030564950 time 759079493626 state 0x2 ref 0 00000100:00100000:2.0:1789760780.017162:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760780.017661:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508448128 00000100:00100000:0.0:1789760780.017672:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508448128 00000100:00100000:0.0:1789760780.017802:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759030564950 time 759079493626 state 0x0 ref 1 00000100:00100000:0.0:1789760780.017819:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759079754042 time 759079493626 state 0x1 ref 1 00000100:00100000:0.0:1789760780.017882:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759079754042 time 759122592474 state 0x0 ref 1 00000100:00100000:0.0:1789760780.017895:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508448128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760780.017943:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24395776, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:0.0:1789760780.017953:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.017969:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760780.017973:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.017980:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.017995:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.030406:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760780.045596:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973276 is committed 00002000:00010000:1.0:1789760780.045667:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24395776 -> 24399872 00010000:00010000:1.0:1789760780.045681:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.045892:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.045903:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.045910:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760780.045914:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.045922:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.045942:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.045949:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe569eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24395776->24399871] (req 24395776->24399871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.046071:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508448128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28173us (28915us total) trans 4294973276 rc 0/0 00000100:00100000:1.0:1789760780.046083:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5988 00000100:00100000:1.0:1789760780.046094:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759079754042 time 759122592474 state 0x2 ref 0 00000100:00100000:1.0:1789760780.051984:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760780.052086:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508448256 00000100:00100000:0.0:1789760780.052095:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508448256 00000100:00100000:0.0:1789760780.052156:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759079754042 time 759122592474 state 0x0 ref 1 00000100:00100000:0.0:1789760780.052171:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759122852890 time 759122592474 state 0x1 ref 1 00000100:00100000:0.0:1789760780.052197:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759122852890 time 759156907979 state 0x0 ref 1 00000100:00100000:0.0:1789760780.052212:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508448256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760780.052259:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24399872, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:0.0:1789760780.052268:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.052316:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760780.052321:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.052328:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.052344:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760780.088062:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24399872 -> 24403968 00010000:00010000:0.0:1789760780.088083:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.088096:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.088105:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.088113:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760780.088117:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.088127:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.088213:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.088222:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe569f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24399872->24403967] (req 24399872->24403967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.088347:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508448256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36130us (36366us total) trans 4294973277 rc 0/0 00000100:00100000:0.0:1789760780.088358:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5989 00000100:00100000:0.0:1789760780.088367:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759122852890 time 759156907979 state 0x2 ref 0 00000001:00080000:2.0:1789760780.088438:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973277 is committed 00000100:00100000:2.0:1789760780.091790:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760780.091992:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508448384 00000100:00100000:0.0:1789760780.092004:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508448384 00000100:00100000:0.0:1789760780.092068:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759122852890 time 759156907979 state 0x0 ref 1 00000100:00100000:0.0:1789760780.092088:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759157168395 time 759156907979 state 0x1 ref 1 00000100:00100000:0.0:1789760780.092236:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759157168395 time 759196945460 state 0x0 ref 1 00000100:00100000:0.0:1789760780.092249:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508448384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760780.092324:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24403968, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:0.0:1789760780.092334:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe569f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.092351:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760780.092355:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.092361:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe569f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.092374:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe569f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.127350:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973278 is committed 00002000:00010000:0.0:1789760780.127424:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24403968 -> 24408064 00010000:00010000:0.0:1789760780.127440:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe569f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.127452:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe569f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.127462:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe569f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.127470:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760780.127473:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe569f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.127480:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe569f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.127496:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe569f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.127501:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe569f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24403968->24408063] (req 24403968->24408063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.127579:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508448384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35329us (35884us total) trans 4294973278 rc 0/0 00000100:00100000:0.0:1789760780.127586:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5990 00000100:00100000:0.0:1789760780.127593:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759157168395 time 759196945460 state 0x2 ref 0 00000100:00100000:2.0:1789760780.134535:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760780.135869:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508448512 00000100:00100000:1.0:1789760780.135882:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508448512 00000100:00100000:1.0:1789760780.135944:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759157168395 time 759196945460 state 0x0 ref 1 00000100:00100000:1.0:1789760780.135961:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759197205876 time 759196945460 state 0x1 ref 1 00000100:00100000:1.0:1789760780.136060:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759197205876 time 759240769198 state 0x0 ref 1 00000100:00100000:1.0:1789760780.136074:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508448512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760780.136131:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24408064, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:1.0:1789760780.136141:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56a00 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.136157:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:1.0:1789760780.136162:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.136169:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56a00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.136183:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56a00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.158435:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760780.158456:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499ca5dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534086784:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760780.158524:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760780.158590:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534086784 00000100:00100000:0.0:1789760780.158598:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534086784 00000100:00100000:0.0:1789760780.158642:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:0.0:1789760780.158653:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534086784:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760780.158689:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:0.0:1789760780.158744:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534086784:12345-0@lo:41:osp-pre-1-0.0 Request processed in 93us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760780.158752:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00100000:1.0:1789760780.158997:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499ca5dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534086784:0@lo:41:osp-pre-1-0.0 00000001:00080000:2.0:1789760780.168068:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973279 is committed 00002000:00010000:1.0:1789760780.168305:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24408064 -> 24412160 00010000:00010000:1.0:1789760780.168321:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56a00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.168340:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56a00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.168350:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56a00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.168358:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760780.168363:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56a00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.168373:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56a00 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.168396:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56a00 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.168406:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56a00 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24408064->24412159] (req 24408064->24412159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.168498:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508448512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32422us (33967us total) trans 4294973279 rc 0/0 00000100:00100000:1.0:1789760780.168509:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5991 00000100:00100000:1.0:1789760780.168519:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759197205876 time 759240769198 state 0x2 ref 0 00000100:00100000:1.0:1789760780.173971:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760780.174278:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508448640 00000100:00100000:0.0:1789760780.175372:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508448640 00000100:00100000:2.0:1789760780.175697:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759197205876 time 759240769198 state 0x0 ref 1 00000100:00100000:2.0:1789760780.175712:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759241029614 time 759240769198 state 0x1 ref 1 00000100:00100000:2.0:1789760780.175751:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759241029614 time 759280462055 state 0x0 ref 1 00000100:00100000:2.0:1789760780.175764:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508448640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760780.175819:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24412160, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:2.0:1789760780.175828:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a07 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.175843:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760780.175848:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.175892:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.175908:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.228448:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973280 is committed 00002000:00010000:2.0:1789760780.228499:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24412160 -> 24416256 00010000:00010000:2.0:1789760780.228514:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.228526:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.228535:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.228541:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760780.228546:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.228555:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a07 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.228576:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a07 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.228661:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a07 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24412160->24416255] (req 24412160->24416255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.228792:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508448640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53017us (54817us total) trans 4294973280 rc 0/0 00000100:00100000:2.0:1789760780.228802:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5992 00000100:00100000:2.0:1789760780.228812:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759241029614 time 759280462055 state 0x2 ref 0 00000100:00100000:2.0:1789760780.235517:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760780.235895:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508448768 00000100:00100000:0.0:1789760780.235905:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508448768 00000100:00100000:0.0:1789760780.235954:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759241029614 time 759280462055 state 0x0 ref 1 00000100:00100000:0.0:1789760780.235972:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759280722471 time 759280462055 state 0x1 ref 1 00000100:00100000:0.0:1789760780.236183:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759280722471 time 759340892585 state 0x0 ref 1 00000100:00100000:0.0:1789760780.236197:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7343b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508448768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760780.236242:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24416256, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:0.0:1789760780.236251:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56a0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.236265:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760780.236270:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.236275:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56a0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.236504:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56a0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.281615:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973281 is committed 00002000:00010000:2.0:1789760780.281999:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24416256 -> 24420352 00010000:00010000:2.0:1789760780.282015:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56a0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.282028:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56a0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.282037:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56a0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.282045:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760780.282051:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56a0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.282060:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56a0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.282079:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56a0e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.282086:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56a0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24416256->24420351] (req 24416256->24420351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.282235:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7343b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508448768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46036us (46723us total) trans 4294973281 rc 0/0 00000100:00100000:2.0:1789760780.282245:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5993 00000100:00100000:2.0:1789760780.282254:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759280722471 time 759340892585 state 0x2 ref 0 00000100:00100000:1.0:1789760780.285574:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760780.285685:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508448896 00000100:00100000:2.0:1789760780.285694:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508448896 00000100:00100000:2.0:1789760780.285914:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759280722471 time 759340892585 state 0x0 ref 1 00000100:00100000:2.0:1789760780.285929:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759341153001 time 759340892585 state 0x1 ref 1 00000100:00100000:2.0:1789760780.285997:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759341153001 time 759390707531 state 0x0 ref 1 00000100:00100000:2.0:1789760780.286010:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508448896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760780.286055:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24420352, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:2.0:1789760780.286064:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.286079:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760780.286083:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.286089:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.286113:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.313465:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973282 is committed 00002000:00010000:2.0:1789760780.314953:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24420352 -> 24424448 00010000:00010000:2.0:1789760780.314971:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.314984:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.314994:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.315002:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760780.315007:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.315016:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.315037:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.315044:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24420352->24424447] (req 24420352->24424447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.315568:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508448896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29554us (29998us total) trans 4294973282 rc 0/0 00000100:00100000:3.0:1789760780.315578:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5994 00000100:00100000:3.0:1789760780.315588:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759341153001 time 759390707531 state 0x2 ref 0 00000100:00100000:1.0:1789760780.323195:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760780.323521:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508449024 00000100:00100000:2.0:1789760780.323535:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508449024 00000100:00100000:2.0:1789760780.323727:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759341153001 time 759390707531 state 0x0 ref 1 00000100:00100000:2.0:1789760780.323747:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759390967947 time 759390707531 state 0x1 ref 1 00000100:00100000:2.0:1789760780.323845:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759390967947 time 759428554112 state 0x0 ref 1 00000100:00100000:2.0:1789760780.323861:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508449024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760780.323930:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24424448, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:2.0:1789760780.323943:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a1c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.323963:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760780.323969:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.323976:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.323994:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.352142:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973283 is committed 00002000:00010000:2.0:1789760780.352239:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24424448 -> 24428544 00010000:00010000:2.0:1789760780.352254:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.352268:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.352276:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.352314:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760780.352319:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.352329:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.352348:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a1c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.352355:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24424448->24428543] (req 24424448->24428543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.352681:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508449024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28583us (29260us total) trans 4294973283 rc 0/0 00000100:00100000:2.0:1789760780.352693:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5995 00000100:00100000:2.0:1789760780.352702:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759390967947 time 759428554112 state 0x2 ref 0 00000100:00100000:2.0:1789760780.360896:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760780.361259:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508449152 00000100:00100000:0.0:1789760780.361270:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508449152 00000100:00100000:0.0:1789760780.361345:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759390967947 time 759428554112 state 0x0 ref 1 00000100:00100000:0.0:1789760780.361359:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759428814528 time 759428554112 state 0x1 ref 1 00000100:00100000:0.0:1789760780.361393:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759428814528 time 759466103438 state 0x0 ref 1 00000100:00100000:0.0:1789760780.361411:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508449152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760780.361454:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24428544, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:0.0:1789760780.361463:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a23 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.361477:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760780.361481:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.361487:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.361502:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.414928:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973284 is committed 00002000:00010000:0.0:1789760780.415115:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24428544 -> 24432640 00010000:00010000:0.0:1789760780.415142:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.415156:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.415165:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.415172:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760780.415177:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.415187:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a23 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.415206:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a23 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.415213:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24428544->24432639] (req 24428544->24432639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.415332:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508449152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53919us (54441us total) trans 4294973284 rc 0/0 00000100:00100000:0.0:1789760780.415343:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5996 00000100:00100000:0.0:1789760780.415353:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759428814528 time 759466103438 state 0x2 ref 0 00000100:00100000:1.0:1789760780.427095:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760780.427189:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508449280 00000100:00100000:3.0:1789760780.427199:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508449280 00000100:00100000:3.0:1789760780.427252:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759428814528 time 759466103438 state 0x0 ref 1 00000100:00100000:3.0:1789760780.427266:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759466363854 time 759466103438 state 0x1 ref 1 00000100:00100000:3.0:1789760780.427333:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759466363854 time 759532043452 state 0x0 ref 1 00000100:00100000:3.0:1789760780.427345:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508449280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760780.427493:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24432640, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:3.0:1789760780.427501:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56a2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.427516:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760780.427520:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.427526:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56a2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.427539:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56a2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.460013:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973285 is committed 00002000:00010000:2.0:1789760780.460156:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24432640 -> 24436736 00010000:00010000:2.0:1789760780.460171:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56a2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.460184:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56a2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.460193:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56a2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.460201:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760780.460206:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56a2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.460216:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56a2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.460236:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56a2a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.460245:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56a2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24432640->24436735] (req 24432640->24436735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.460369:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508449280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33020us (33280us total) trans 4294973285 rc 0/0 00000100:00100000:2.0:1789760780.460380:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5997 00000100:00100000:2.0:1789760780.460390:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759466363854 time 759532043452 state 0x2 ref 0 00000100:00100000:1.0:1789760780.469379:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760780.469500:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508449408 00000100:00100000:3.0:1789760780.469511:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508449408 00000100:00100000:3.0:1789760780.469572:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759466363854 time 759532043452 state 0x0 ref 1 00000100:00100000:3.0:1789760780.469587:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759532303868 time 759532043452 state 0x1 ref 1 00000100:00100000:3.0:1789760780.469622:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759532303868 time 759574333170 state 0x0 ref 1 00000100:00100000:3.0:1789760780.469636:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508449408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760780.469685:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24436736, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0xc007 00010000:00010000:3.0:1789760780.469694:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56a31 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.469711:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760780.469716:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.469722:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56a31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.469737:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56a31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.515201:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973286 is committed 00002000:00010000:2.0:1789760780.515943:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24436736 -> 24440832 00002000:00010000:2.0:1789760780.515949:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760780 -> 1789760781 00002000:00010000:2.0:1789760780.515952:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760780 -> 1789760781 00010000:00010000:2.0:1789760780.516121:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56a31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.516135:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56a31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.516144:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56a31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.516158:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760780.516163:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56a31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.516172:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56a31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.516193:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56a31 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.516201:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56a31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24436736->24440831] (req 24436736->24440831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.516394:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508449408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46755us (47020us total) trans 4294973286 rc 0/0 00000100:00100000:2.0:1789760780.516404:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5998 00000100:00100000:2.0:1789760780.516414:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759532303868 time 759574333170 state 0x2 ref 0 00000100:00100000:3.0:1789760780.523085:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760780.524244:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508449536 00000100:00100000:0.0:1789760780.524258:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508449536 00000100:00100000:0.0:1789760780.524493:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759532303868 time 759574333170 state 0x0 ref 1 00000100:00100000:0.0:1789760780.524509:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759574593586 time 759574333170 state 0x1 ref 1 00000100:00100000:0.0:1789760780.524546:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759574593586 time 759629255951 state 0x0 ref 1 00000100:00100000:0.0:1789760780.524560:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508449536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760780.524614:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24440832, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:0.0:1789760780.524624:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.524642:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:0.0:1789760780.524646:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.524652:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.524668:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.549418:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973287 is committed 00002000:00010000:2.0:1789760780.549524:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24440832 -> 24444928 00010000:00010000:2.0:1789760780.549539:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.549553:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.549568:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.549576:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760780.549581:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.549590:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.549609:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a38 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.549618:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56a38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24440832->24444927] (req 24440832->24444927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.549708:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508449536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25146us (26628us total) trans 4294973287 rc 0/0 00000100:00100000:2.0:1789760780.549718:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 5999 00000100:00100000:2.0:1789760780.549727:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759574593586 time 759629255951 state 0x2 ref 0 00000100:00100000:1.0:1789760780.553934:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760780.554627:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508449664 00000100:00100000:2.0:1789760780.554637:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508449664 00000100:00100000:2.0:1789760780.554693:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759574593586 time 759629255951 state 0x0 ref 1 00000100:00100000:2.0:1789760780.554708:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759629516367 time 759629255951 state 0x1 ref 1 00000100:00100000:2.0:1789760780.554779:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759629516367 time 759659487791 state 0x0 ref 1 00000100:00100000:2.0:1789760780.554802:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508449664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760780.554849:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24444928, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:2.0:1789760780.554858:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a3f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.554872:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760780.554877:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.554884:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.554899:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760780.618589:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24444928 -> 24449024 00010000:00010000:2.0:1789760780.618609:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.618622:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.618630:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.618638:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760780.618643:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.618651:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.618671:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a3f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.618680:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24444928->24449023] (req 24444928->24449023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.619082:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508449664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 64273us (65149us total) trans 4294973288 rc 0/0 00000100:00100000:2.0:1789760780.619094:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6000 00000100:00100000:2.0:1789760780.619104:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759629516367 time 759659487791 state 0x2 ref 0 00000001:00080000:3.0:1789760780.619962:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973288 is committed 00000100:00100000:2.0:1789760780.626146:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760780.626540:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508449792 00000100:00100000:3.0:1789760780.626558:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508449792 00000100:00100000:3.0:1789760780.626639:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759629516367 time 759659487791 state 0x0 ref 1 00000100:00100000:3.0:1789760780.626660:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759659748207 time 759659487791 state 0x1 ref 1 00000100:00100000:3.0:1789760780.627388:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759659748207 time 759732083576 state 0x0 ref 1 00000100:00100000:2.0:1789760780.627883:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508449792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760780.627941:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24449024, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:2.0:1789760780.627952:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a46 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.627968:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760780.627973:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.627979:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.627993:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760780.694538:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24449024 -> 24453120 00010000:00010000:2.0:1789760780.694560:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.694571:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a46 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.694578:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.694583:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760780.694588:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.694595:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a46 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.694615:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a46 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.694620:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a46 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24449024->24453119] (req 24449024->24453119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.694752:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508449792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 67340us (68608us total) trans 4294973289 rc 0/0 00000001:00080000:0.0:1789760780.694763:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973289 is committed 00000100:00100000:2.0:1789760780.694765:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6001 00000100:00100000:2.0:1789760780.694778:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759659748207 time 759732083576 state 0x2 ref 0 00000100:00100000:1.0:1789760780.701421:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760780.702151:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508449920 00000100:00100000:2.0:1789760780.702167:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508449920 00000100:00100000:2.0:1789760780.702249:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759659748207 time 759732083576 state 0x0 ref 1 00000100:00100000:2.0:1789760780.702269:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759732343992 time 759732083576 state 0x1 ref 1 00000100:00100000:3.0:1789760780.702546:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759732343992 time 759807252243 state 0x0 ref 1 00000100:00100000:3.0:1789760780.702566:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508449920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760780.702631:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24453120, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:3.0:1789760780.702643:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56a4d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.702813:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:3.0:1789760780.702818:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.702826:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56a4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.702844:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56a4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.763088:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973290 is committed 00002000:00010000:2.0:1789760780.763210:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24453120 -> 24457216 00010000:00010000:2.0:1789760780.763229:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56a4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.763244:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56a4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.763254:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56a4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.763262:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760780.763268:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56a4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.763278:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56a4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.763416:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56a4d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.763425:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56a4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24453120->24457215] (req 24453120->24457215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.763541:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508449920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60975us (62129us total) trans 4294973290 rc 0/0 00000100:00100000:2.0:1789760780.763551:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6002 00000100:00100000:2.0:1789760780.763563:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759732343992 time 759807252243 state 0x2 ref 0 00000100:00100000:1.0:1789760780.767406:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760780.767505:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508450048 00000100:00100000:2.0:1789760780.767518:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508450048 00000100:00100000:2.0:1789760780.767591:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759732343992 time 759807252243 state 0x0 ref 1 00000100:00100000:2.0:1789760780.767609:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759807512659 time 759807252243 state 0x1 ref 1 00000100:00100000:2.0:1789760780.767643:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759807512659 time 759872352954 state 0x0 ref 1 00000100:00100000:2.0:1789760780.767657:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508450048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760780.767732:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24457216, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:2.0:1789760780.767743:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.767763:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760780.767769:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.767778:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.767805:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.810661:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973291 is committed 00002000:00010000:0.0:1789760780.810858:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24457216 -> 24461312 00010000:00010000:0.0:1789760780.810880:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.810895:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.810906:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.810913:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760780.810919:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.810930:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.810961:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.810970:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56a54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24457216->24461311] (req 24457216->24461311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.811097:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508450048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43435us (43698us total) trans 4294973291 rc 0/0 00000100:00100000:0.0:1789760780.811110:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6003 00000100:00100000:0.0:1789760780.811123:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759807512659 time 759872352954 state 0x2 ref 0 00000100:00100000:1.0:1789760780.815919:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760780.819568:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508450176 00000100:00100000:2.0:1789760780.819588:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508450176 00000100:00100000:2.0:1789760780.819674:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759807512659 time 759872352954 state 0x0 ref 1 00000100:00100000:2.0:1789760780.819694:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759872613370 time 759872352954 state 0x1 ref 1 00000100:00100000:2.0:1789760780.819764:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759872613370 time 759924472948 state 0x0 ref 1 00000100:00100000:2.0:1789760780.819788:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508450176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760780.819851:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24461312, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:2.0:1789760780.819862:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56a5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.819881:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760780.819886:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.819893:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56a5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.819909:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56a5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760780.872399:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24461312 -> 24465408 00010000:00010000:3.0:1789760780.872425:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56a5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.872440:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56a5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.872449:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56a5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.872457:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760780.872463:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56a5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.872474:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56a5b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.872500:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56a5b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.872509:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973292 is committed 00010000:00010000:3.0:1789760780.872509:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56a5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24461312->24465407] (req 24461312->24465407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.872672:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508450176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52888us (56760us total) trans 4294973292 rc 0/0 00000100:00100000:3.0:1789760780.872685:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6004 00000100:00100000:3.0:1789760780.872698:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759872613370 time 759924472948 state 0x2 ref 0 00000100:00100000:1.0:1789760780.876409:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760780.876532:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508450304 00000100:00100000:2.0:1789760780.876547:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508450304 00000100:00100000:2.0:1789760780.876624:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759872613370 time 759924472948 state 0x0 ref 1 00000100:00100000:2.0:1789760780.876642:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759924733364 time 759924472948 state 0x1 ref 1 00000100:00100000:2.0:1789760780.876679:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759924733364 time 759981389280 state 0x0 ref 1 00000100:00100000:2.0:1789760780.876999:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508450304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760780.877083:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24465408, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:2.0:1789760780.877094:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56a62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.877115:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=759s v=(0 0) 00010000:00010000:2.0:1789760780.877121:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.877253:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56a62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.877275:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56a62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.926959:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973293 is committed 00002000:00010000:3.0:1789760780.927265:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24465408 -> 24469504 00010000:00010000:3.0:1789760780.927320:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56a62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.927337:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56a62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.927347:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56a62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.927357:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760780.927362:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56a62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.927373:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56a62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.927413:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56a62 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.927422:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56a62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24465408->24469503] (req 24465408->24469503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.927554:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508450304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50855us (51153us total) trans 4294973293 rc 0/0 00000100:00100000:3.0:1789760780.927568:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6005 00000100:00100000:3.0:1789760780.927580:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759924733364 time 759981389280 state 0x2 ref 0 00000100:00100000:1.0:1789760780.930880:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760780.930934:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508450432 00000100:00100000:0.0:1789760780.930946:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508450432 00000100:00100000:0.0:1789760780.931018:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759924733364 time 759981389280 state 0x0 ref 1 00000100:00100000:0.0:1789760780.931036:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759981649696 time 759981389280 state 0x1 ref 1 00000100:00100000:0.0:1789760780.931079:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759981649696 time 760035789170 state 0x0 ref 1 00000100:00100000:0.0:1789760780.931094:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508450432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760780.931154:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24469504, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:0.0:1789760780.931166:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56a69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.931191:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760780.931196:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.931204:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56a69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.931219:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56a69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.974749:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973294 is committed 00002000:00010000:2.0:1789760780.974894:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24469504 -> 24473600 00010000:00010000:2.0:1789760780.974915:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56a69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.974938:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56a69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.974948:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56a69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.974955:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760780.974961:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56a69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.974971:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56a69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.974998:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56a69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.975007:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56a69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24469504->24473599] (req 24469504->24473599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.975979:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508450432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44035us (44260us total) trans 4294973294 rc 0/0 00000100:00100000:2.0:1789760780.975995:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6006 00000100:00100000:2.0:1789760780.976008:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759981649696 time 760035789170 state 0x2 ref 0 00000100:00100000:1.0:1789760780.982355:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760780.982645:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508450560 00000100:00100000:3.0:1789760780.982658:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508450560 00000100:00100000:2.0:1789760780.982964:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 759981649696 time 760035789170 state 0x0 ref 1 00000100:00100000:2.0:1789760780.982985:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760036049586 time 760035789170 state 0x1 ref 1 00000100:00100000:2.0:1789760780.983059:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760036049586 time 760087768504 state 0x0 ref 1 00000100:00100000:2.0:1789760780.983074:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508450560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760780.983146:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24473600, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:2.0:1789760780.983157:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.983176:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760780.983180:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.983188:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.983205:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.025053:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973295 is committed 00002000:00010000:2.0:1789760781.025197:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24473600 -> 24477696 00010000:00010000:2.0:1789760781.025226:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.025243:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.025253:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.025267:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760781.025274:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.025324:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a70 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.025351:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a70 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.025361:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24473600->24477695] (req 24473600->24477695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.025621:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508450560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42541us (43273us total) trans 4294973295 rc 0/0 00000100:00100000:2.0:1789760781.025635:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6007 00000100:00100000:2.0:1789760781.025648:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760036049586 time 760087768504 state 0x2 ref 0 00000100:00100000:3.0:1789760781.028217:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.028321:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508450688 00000100:00100000:2.0:1789760781.028334:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508450688 00000100:00100000:2.0:1789760781.028430:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760036049586 time 760087768504 state 0x0 ref 1 00000100:00100000:2.0:1789760781.028450:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760088028920 time 760087768504 state 0x1 ref 1 00000100:00100000:2.0:1789760781.028477:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760088028920 time 760133188088 state 0x0 ref 1 00000100:00100000:2.0:1789760781.028495:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508450688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760781.028558:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24477696, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:2.0:1789760781.028568:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.028587:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.028592:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.028601:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.028617:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760781.028946:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760781.054874:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973296 is committed 00010000:00010000:1.0:1789760781.055725:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760781.055971:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24477696 -> 24481792 00010000:00010000:2.0:1789760781.055990:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.056006:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.056017:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.056025:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760781.056030:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.056043:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.056067:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.056075:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24477696->24481791] (req 24477696->24481791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.056357:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508450688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27857us (28148us total) trans 4294973296 rc 0/0 00000100:00100000:2.0:1789760781.056370:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6008 00000100:00100000:2.0:1789760781.056384:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760088028920 time 760133188088 state 0x2 ref 0 00000100:00100000:3.0:1789760781.059918:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.060086:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508450816 00000100:00100000:2.0:1789760781.060100:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508450816 00000100:00100000:2.0:1789760781.060393:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760088028920 time 760133188088 state 0x0 ref 1 00000100:00100000:2.0:1789760781.060413:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760133448504 time 760133188088 state 0x1 ref 1 00000100:00100000:0.0:1789760781.060491:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760133448504 time 760165197722 state 0x0 ref 1 00000100:00100000:0.0:1789760781.060519:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508450816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760781.060592:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24481792, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:0.0:1789760781.060603:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56a7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.060623:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760781.060628:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.060636:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56a7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.060654:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56a7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760781.097932:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24481792 -> 24485888 00010000:00010000:3.0:1789760781.097957:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56a7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.097972:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56a7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.097981:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56a7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.097989:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760781.097995:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56a7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.098004:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56a7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.098028:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56a7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.098036:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56a7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24481792->24485887] (req 24481792->24485887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.098180:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973297 is committed 00000100:00100000:3.0:1789760781.098362:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508450816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37848us (38449us total) trans 4294973297 rc 0/0 00000100:00100000:3.0:1789760781.098375:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6009 00000100:00100000:3.0:1789760781.098387:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760133448504 time 760165197722 state 0x2 ref 0 00000100:00100000:1.0:1789760781.101821:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.102004:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508450944 00000100:00100000:2.0:1789760781.102018:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508450944 00000100:00100000:2.0:1789760781.102093:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760133448504 time 760165197722 state 0x0 ref 1 00000100:00100000:2.0:1789760781.102112:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760165458138 time 760165197722 state 0x1 ref 1 00000100:00100000:2.0:1789760781.102153:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760165458138 time 760206862829 state 0x0 ref 1 00000100:00100000:2.0:1789760781.102174:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508450944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760781.102238:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24485888, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:2.0:1789760781.102249:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.102270:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.102274:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.102282:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.102344:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.147318:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973298 is committed 00002000:00010000:2.0:1789760781.147478:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24485888 -> 24489984 00010000:00010000:2.0:1789760781.147497:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.147511:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.147521:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.147529:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760781.147598:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.147612:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.147638:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.147645:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24485888->24489983] (req 24485888->24489983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.147904:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508450944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45652us (46019us total) trans 4294973298 rc 0/0 00000100:00100000:2.0:1789760781.147918:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6010 00000100:00100000:2.0:1789760781.147929:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760165458138 time 760206862829 state 0x2 ref 0 00000100:00100000:1.0:1789760781.154083:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.155752:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508451072 00000100:00100000:2.0:1789760781.155771:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508451072 00000100:00100000:2.0:1789760781.155874:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760165458138 time 760206862829 state 0x0 ref 1 00000100:00100000:2.0:1789760781.155894:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760207123245 time 760206862829 state 0x1 ref 1 00000100:00100000:2.0:1789760781.155934:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760207123245 time 760260643313 state 0x0 ref 1 00000100:00100000:2.0:1789760781.155950:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508451072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760781.156016:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24489984, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:2.0:1789760781.156028:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.156049:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.156055:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.156063:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.156079:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.188656:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973299 is committed 00002000:00010000:0.0:1789760781.188806:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24489984 -> 24494080 00010000:00010000:0.0:1789760781.188826:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.188841:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.188850:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.188858:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760781.188863:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.188872:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a8c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.188898:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a8c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.188906:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56a8c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24489984->24494079] (req 24489984->24494079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.189039:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508451072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33085us (34967us total) trans 4294973299 rc 0/0 00000100:00100000:0.0:1789760781.189052:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6011 00000100:00100000:0.0:1789760781.189065:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760207123245 time 760260643313 state 0x2 ref 0 00000100:00100000:3.0:1789760781.194886:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760781.195139:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508451200 00000100:00100000:0.0:1789760781.195153:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508451200 00000100:00100000:0.0:1789760781.195237:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760207123245 time 760260643313 state 0x0 ref 1 00000100:00100000:0.0:1789760781.195256:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760260903729 time 760260643313 state 0x1 ref 1 00000100:00100000:0.0:1789760781.195309:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760260903729 time 760300018723 state 0x0 ref 1 00000100:00100000:0.0:1789760781.195328:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508451200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760781.195391:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24494080, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:0.0:1789760781.195402:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56a93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.195421:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760781.195426:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.195434:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56a93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.195450:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56a93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760781.235310:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24494080 -> 24498176 00010000:00010000:3.0:1789760781.235339:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56a93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.235355:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56a93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.235366:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56a93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.235374:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760781.235380:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56a93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.235389:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56a93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.235414:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56a93 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.235424:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56a93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24494080->24498175] (req 24494080->24498175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.235562:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508451200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40229us (41636us total) trans 4294973300 rc 0/0 00000100:00100000:3.0:1789760781.235574:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6012 00000100:00100000:3.0:1789760781.235587:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760260903729 time 760300018723 state 0x2 ref 0 00000001:00080000:1.0:1789760781.236434:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973300 is committed 00000100:00100000:1.0:1789760781.238476:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.238569:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508451328 00000100:00100000:2.0:1789760781.238582:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508451328 00000100:00100000:2.0:1789760781.238652:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760260903729 time 760300018723 state 0x0 ref 1 00000100:00100000:2.0:1789760781.238671:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760300279139 time 760300018723 state 0x1 ref 1 00000100:00100000:2.0:1789760781.238713:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760300279139 time 760343422023 state 0x0 ref 1 00000100:00100000:2.0:1789760781.238728:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d41c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508451328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760781.238802:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24498176, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:2.0:1789760781.238814:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a9a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.238832:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.238837:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.238845:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.238867:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.280236:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973301 is committed 00002000:00010000:0.0:1789760781.280473:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24498176 -> 24502272 00010000:00010000:0.0:1789760781.280494:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.280508:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.280519:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.280527:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760781.280533:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.280543:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.280569:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.280577:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56a9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24498176->24502271] (req 24498176->24502271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.280838:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d41c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508451328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42105us (42367us total) trans 4294973301 rc 0/0 00000100:00100000:0.0:1789760781.280850:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6013 00000100:00100000:0.0:1789760781.280864:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760300279139 time 760343422023 state 0x2 ref 0 00000100:00100000:0.0:1789760781.285978:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.286428:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508451456 00000100:00100000:2.0:1789760781.286440:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508451456 00000100:00100000:2.0:1789760781.286506:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760300279139 time 760343422023 state 0x0 ref 1 00000100:00100000:2.0:1789760781.286522:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760343682439 time 760343422023 state 0x1 ref 1 00000100:00100000:2.0:1789760781.286552:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760343682439 time 760391262416 state 0x0 ref 1 00000100:00100000:2.0:1789760781.286566:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508451456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760781.286642:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24502272, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:2.0:1789760781.286745:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56aa1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.286765:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.286771:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.286786:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56aa1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.286804:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56aa1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760781.328971:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24502272 -> 24506368 00010000:00010000:0.0:1789760781.328999:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56aa1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.329021:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56aa1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.329032:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56aa1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.329041:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760781.329047:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56aa1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.329058:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56aa1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.329084:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56aa1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.329093:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56aa1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24502272->24506367] (req 24502272->24506367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.329233:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508451456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42661us (43261us total) trans 4294973302 rc 0/0 00000100:00100000:0.0:1789760781.329244:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6014 00000100:00100000:0.0:1789760781.329256:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760343682439 time 760391262416 state 0x2 ref 0 00000001:00080000:3.0:1789760781.329901:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973302 is committed 00000100:00100000:1.0:1789760781.336179:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760781.336353:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508451584 00000100:00100000:3.0:1789760781.336368:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508451584 00000100:00100000:3.0:1789760781.336448:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760343682439 time 760391262416 state 0x0 ref 1 00000100:00100000:3.0:1789760781.336470:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760391522832 time 760391262416 state 0x1 ref 1 00000100:00100000:3.0:1789760781.336509:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760391522832 time 760441218428 state 0x0 ref 1 00000100:00100000:3.0:1789760781.336523:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508451584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760781.336587:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24506368, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:3.0:1789760781.336598:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56aa8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.336618:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760781.336624:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.336631:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56aa8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.336649:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56aa8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.383634:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760781.383688:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760781.383729:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760781.383753:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760781.383790:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760781.383826:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508451840 00000100:00100000:0.0:1789760781.383838:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508451840 00000100:00100000:0.0:1789760781.383909:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508452096 00000100:00100000:0.0:1789760781.383913:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508452096 00000100:00100000:0.0:1789760781.383932:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508451968 00000100:00100000:0.0:1789760781.383935:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508451968 00000100:00100000:0.0:1789760781.383948:0:8128:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508452224 00000100:00100000:0.0:1789760781.383951:0:8128:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508452224 00000100:00100000:0.0:1789760781.383968:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 969 00000100:00100000:0.0:1789760781.383980:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2407:x1876699508451840:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760781.384077:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2407:x1876699508451840:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 99us (459us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760781.384085:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 969 00000100:00100000:0.0:1789760781.384104:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 970 00000100:00100000:0.0:1789760781.384110:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508452096:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760781.384150:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2407:x1876699508452096:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 40us (462us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760781.384157:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 970 00000100:00100000:0.0:1789760781.384178:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 971 00000100:00100000:0.0:1789760781.384185:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508451968:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760781.384212:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508451968:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 26us (457us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760781.384218:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 971 00000100:00100000:0.0:1789760781.384230:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 972 00000100:00100000:0.0:1789760781.384236:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2406:x1876699508452224:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760781.384259:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2406:x1876699508452224:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 24us (469us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760781.384265:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 972 00000100:00080000:0.0:1789760781.384320:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789760781.384325:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760781.384499:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508451712 00000100:00100000:0.0:1789760781.384505:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508451712 00000100:00100000:0.0:1789760781.384534:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 487 00000100:00100000:0.0:1789760781.384541:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988ca9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2406:x1876699508451712:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760781.384577:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988ca9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2406:x1876699508451712:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 37us (847us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760781.384584:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 487 00000001:00080000:0.0:1789760781.390356:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973303 is committed 00002000:00010000:3.0:1789760781.390498:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24506368 -> 24510464 00010000:00010000:3.0:1789760781.390518:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56aa8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.390534:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56aa8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.391030:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56aa8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.391050:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760781.391056:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56aa8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.391067:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56aa8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.391096:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56aa8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.391106:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56aa8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24506368->24510463] (req 24506368->24510463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.391937:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508451584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55408us (55776us total) trans 4294973303 rc 0/0 00000100:00100000:3.0:1789760781.391952:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6015 00000100:00100000:3.0:1789760781.391965:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760391522832 time 760441218428 state 0x2 ref 0 00000100:00100000:0.0:1789760781.395953:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.396063:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508452352 00000100:00100000:2.0:1789760781.396076:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508452352 00000100:00100000:2.0:1789760781.396216:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760391522832 time 760441218428 state 0x0 ref 1 00000100:00100000:2.0:1789760781.396235:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760441478844 time 760441218428 state 0x1 ref 1 00000100:00100000:0.0:1789760781.396521:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760441478844 time 760501227616 state 0x0 ref 1 00000100:00100000:0.0:1789760781.396538:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508452352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760781.396606:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24510464, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:0.0:1789760781.396618:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56aaf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.396639:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760781.396643:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.396652:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56aaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.396670:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56aaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.432213:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973304 is committed 00002000:00010000:0.0:1789760781.432428:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24510464 -> 24514560 00010000:00010000:0.0:1789760781.432446:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56aaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.432458:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56aaf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.432465:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56aaf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.432470:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760781.432474:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56aaf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.432481:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56aaf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.432500:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56aaf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.432505:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56aaf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24510464->24514559] (req 24510464->24514559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.432610:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508452352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36071us (36669us total) trans 4294973304 rc 0/0 00000100:00100000:0.0:1789760781.432619:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6016 00000100:00100000:0.0:1789760781.432630:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760441478844 time 760501227616 state 0x2 ref 0 00000100:00100000:0.0:1789760781.437036:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.437471:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508452480 00000100:00100000:2.0:1789760781.437485:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508452480 00000100:00100000:2.0:1789760781.437557:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760441478844 time 760501227616 state 0x0 ref 1 00000100:00100000:2.0:1789760781.437575:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760501488032 time 760501227616 state 0x1 ref 1 00000100:00100000:2.0:1789760781.437611:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760501488032 time 760542320623 state 0x0 ref 1 00000100:00100000:2.0:1789760781.437625:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508452480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760781.437698:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24514560, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:2.0:1789760781.437711:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ab6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.437732:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.437737:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.437745:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ab6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.437761:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ab6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.470458:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973305 is committed 00002000:00010000:2.0:1789760781.472338:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24514560 -> 24518656 00010000:00010000:2.0:1789760781.472366:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ab6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.472383:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ab6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.472393:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ab6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.472402:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760781.472407:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ab6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.472419:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ab6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.472444:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ab6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.472453:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ab6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24514560->24518655] (req 24514560->24518655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.472583:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508452480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34954us (35556us total) trans 4294973305 rc 0/0 00000100:00100000:2.0:1789760781.472596:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6017 00000100:00100000:2.0:1789760781.472607:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760501488032 time 760542320623 state 0x2 ref 0 00000100:00100000:1.0:1789760781.477052:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760781.477585:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508452608 00000100:00100000:3.0:1789760781.477599:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508452608 00000100:00100000:3.0:1789760781.477684:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760501488032 time 760542320623 state 0x0 ref 1 00000100:00100000:3.0:1789760781.477765:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760542581039 time 760542320623 state 0x1 ref 1 00000100:00100000:3.0:1789760781.477840:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760542581039 time 760582548971 state 0x0 ref 1 00000100:00100000:3.0:1789760781.477855:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508452608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760781.477920:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24518656, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0xc007 00010000:00010000:3.0:1789760781.477930:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56abd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.477952:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:3.0:1789760781.477958:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.477966:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56abd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.477984:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56abd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760781.507163:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24518656 -> 24522752 00002000:00010000:1.0:1789760781.507174:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760781 -> 1789760782 00002000:00010000:1.0:1789760781.507177:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760781 -> 1789760782 00010000:00010000:1.0:1789760781.507194:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56abd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.507216:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56abd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.507225:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56abd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.507232:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760781.507238:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56abd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.507248:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56abd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.507272:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56abd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.507281:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56abd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24518656->24522751] (req 24518656->24522751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.507420:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508452608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29562us (30376us total) trans 4294973306 rc 0/0 00000100:00100000:1.0:1789760781.507432:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6018 00000100:00100000:1.0:1789760781.507444:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760542581039 time 760582548971 state 0x2 ref 0 00000001:00080000:2.0:1789760781.508362:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973306 is committed 00000100:00100000:1.0:1789760781.512407:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760781.512500:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508452736 00000100:00100000:0.0:1789760781.512514:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508452736 00000100:00100000:0.0:1789760781.512586:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760542581039 time 760582548971 state 0x0 ref 1 00000100:00100000:0.0:1789760781.512606:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760582809387 time 760582548971 state 0x1 ref 1 00000100:00100000:0.0:1789760781.512645:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760582809387 time 760617354522 state 0x0 ref 1 00000100:00100000:0.0:1789760781.512659:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508452736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760781.512736:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24522752, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:0.0:1789760781.512747:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56ac4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.512768:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760781.512783:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.512791:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56ac4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.512809:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56ac4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760781.543722:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24522752 -> 24526848 00010000:00010000:2.0:1789760781.543752:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56ac4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.543769:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56ac4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.543789:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56ac4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.543797:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760781.543803:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56ac4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.543812:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56ac4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.543837:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56ac4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.543845:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56ac4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24522752->24526847] (req 24522752->24526847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.543971:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508452736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31307us (31570us total) trans 4294973307 rc 0/0 00000100:00100000:2.0:1789760781.543985:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6019 00000100:00100000:2.0:1789760781.543998:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760582809387 time 760617354522 state 0x2 ref 0 00000001:00080000:1.0:1789760781.544091:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973307 is committed 00000100:00100000:0.0:1789760781.547665:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.547833:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508452864 00000100:00100000:2.0:1789760781.547847:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508452864 00000100:00100000:2.0:1789760781.547919:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760582809387 time 760617354522 state 0x0 ref 1 00000100:00100000:2.0:1789760781.547938:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760617614938 time 760617354522 state 0x1 ref 1 00000100:00100000:2.0:1789760781.548010:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760617614938 time 760652719027 state 0x0 ref 1 00000100:00100000:2.0:1789760781.548025:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508452864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760781.548093:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24526848, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:2.0:1789760781.548103:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56acb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.548123:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.548129:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.548136:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56acb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.548153:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56acb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.580464:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973308 is committed 00002000:00010000:2.0:1789760781.580626:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24526848 -> 24530944 00010000:00010000:2.0:1789760781.580646:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56acb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.580661:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56acb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.580671:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56acb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.580678:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760781.580685:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56acb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.580694:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56acb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.580717:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56acb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.580724:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56acb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24526848->24530943] (req 24526848->24530943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.580856:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508452864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32826us (33199us total) trans 4294973308 rc 0/0 00000100:00100000:2.0:1789760781.580868:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6020 00000100:00100000:2.0:1789760781.580880:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760617614938 time 760652719027 state 0x2 ref 0 00000100:00100000:3.0:1789760781.584192:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760781.584425:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508452992 00000100:00100000:1.0:1789760781.584439:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508452992 00000100:00100000:1.0:1789760781.584508:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760617614938 time 760652719027 state 0x0 ref 1 00000100:00100000:1.0:1789760781.584527:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760652979443 time 760652719027 state 0x1 ref 1 00000100:00100000:1.0:1789760781.584564:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760652979443 time 760689274773 state 0x0 ref 1 00000100:00100000:1.0:1789760781.584576:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508452992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760781.584635:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24530944, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:1.0:1789760781.584647:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56ad2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.584666:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760781.584672:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.584680:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56ad2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.584697:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56ad2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.617395:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973309 is committed 00002000:00010000:3.0:1789760781.619374:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24530944 -> 24535040 00010000:00010000:3.0:1789760781.619400:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56ad2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.619417:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56ad2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.619427:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56ad2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.619441:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760781.619447:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56ad2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.619457:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56ad2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.619483:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56ad2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.619492:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56ad2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24530944->24535039] (req 24530944->24535039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.619619:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508452992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35037us (35434us total) trans 4294973309 rc 0/0 00000100:00100000:3.0:1789760781.619633:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6021 00000100:00100000:3.0:1789760781.619646:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760652979443 time 760689274773 state 0x2 ref 0 00000100:00100000:3.0:1789760781.623037:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.623594:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508453120 00000100:00100000:2.0:1789760781.623608:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508453120 00000100:00100000:2.0:1789760781.623684:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760652979443 time 760689274773 state 0x0 ref 1 00000100:00100000:2.0:1789760781.623708:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760689535189 time 760689274773 state 0x1 ref 1 00000100:00100000:2.0:1789760781.623753:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760689535189 time 760728462703 state 0x0 ref 1 00000100:00100000:2.0:1789760781.623768:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508453120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760781.623842:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24535040, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:2.0:1789760781.623854:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ad9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.623874:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.623884:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.623892:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ad9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.623909:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ad9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.650801:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973310 is committed 00002000:00010000:2.0:1789760781.651653:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24535040 -> 24539136 00010000:00010000:2.0:1789760781.651676:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ad9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.651690:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ad9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.651700:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ad9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.651713:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760781.651718:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ad9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.651729:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ad9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.651758:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ad9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.651767:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ad9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24535040->24539135] (req 24535040->24539135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.652036:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508453120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28264us (29008us total) trans 4294973310 rc 0/0 00000100:00100000:2.0:1789760781.652048:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6022 00000100:00100000:2.0:1789760781.652059:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760689535189 time 760728462703 state 0x2 ref 0 00000100:00100000:1.0:1789760781.657501:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.657557:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508453248 00000100:00100000:2.0:1789760781.657573:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508453248 00000100:00100000:2.0:1789760781.657650:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760689535189 time 760728462703 state 0x0 ref 1 00000100:00100000:2.0:1789760781.657671:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760728723119 time 760728462703 state 0x1 ref 1 00000100:00100000:2.0:1789760781.657710:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760728723119 time 760762419007 state 0x0 ref 1 00000100:00100000:2.0:1789760781.657730:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508453248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760781.657814:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24539136, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:2.0:1789760781.657834:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ae0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.657854:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.657859:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.657866:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ae0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.657884:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ae0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.681650:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973311 is committed 00002000:00010000:2.0:1789760781.681828:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24539136 -> 24543232 00010000:00010000:2.0:1789760781.681848:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ae0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.681864:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ae0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.681874:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ae0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.681882:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760781.681888:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ae0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.681899:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ae0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.681925:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ae0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.681930:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56ae0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24539136->24543231] (req 24539136->24543231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.682057:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508453248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24323us (24564us total) trans 4294973311 rc 0/0 00000100:00100000:2.0:1789760781.682069:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6023 00000100:00100000:2.0:1789760781.682080:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760728723119 time 760762419007 state 0x2 ref 0 00000100:00100000:3.0:1789760781.685163:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760781.686070:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508453376 00000100:00100000:1.0:1789760781.686088:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508453376 00000100:00100000:1.0:1789760781.686231:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760728723119 time 760762419007 state 0x0 ref 1 00000100:00100000:1.0:1789760781.686251:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760762679423 time 760762419007 state 0x1 ref 1 00000100:00100000:0.0:1789760781.686375:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760762679423 time 760791082002 state 0x0 ref 1 00000100:00100000:0.0:1789760781.686394:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508453376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760781.686459:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24543232, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:0.0:1789760781.686470:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56ae7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.686491:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:0.0:1789760781.686496:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.686503:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56ae7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.686522:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56ae7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.720394:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973312 is committed 00002000:00010000:1.0:1789760781.720594:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24543232 -> 24547328 00010000:00010000:1.0:1789760781.720614:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56ae7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.720631:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56ae7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.720641:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56ae7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.720650:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760781.720655:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56ae7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.720666:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56ae7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.720690:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56ae7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.720696:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56ae7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24543232->24547327] (req 24543232->24547327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.721122:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508453376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34726us (35966us total) trans 4294973312 rc 0/0 00000100:00100000:1.0:1789760781.721136:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6024 00000100:00100000:1.0:1789760781.721149:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760762679423 time 760791082002 state 0x2 ref 0 00000100:00100000:3.0:1789760781.723905:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.724906:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508453504 00000100:00100000:2.0:1789760781.724923:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508453504 00000100:00100000:2.0:1789760781.725072:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760762679423 time 760791082002 state 0x0 ref 1 00000100:00100000:2.0:1789760781.725094:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760791342418 time 760791082002 state 0x1 ref 1 00000100:00100000:2.0:1789760781.725187:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760791342418 time 760829894751 state 0x0 ref 1 00000100:00100000:2.0:1789760781.725203:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508453504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760781.725271:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24547328, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:2.0:1789760781.725282:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56aee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.725342:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.725347:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.725355:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56aee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.725372:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56aee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.759795:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973313 is committed 00002000:00010000:2.0:1789760781.759878:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24547328 -> 24551424 00010000:00010000:2.0:1789760781.760033:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56aee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.760047:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56aee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.760056:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56aee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.760063:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760781.760067:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56aee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.760075:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56aee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.760095:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56aee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.760103:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56aee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24547328->24551423] (req 24547328->24551423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.760189:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508453504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34985us (36296us total) trans 4294973313 rc 0/0 00000100:00100000:2.0:1789760781.760199:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6025 00000100:00100000:2.0:1789760781.760209:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760791342418 time 760829894751 state 0x2 ref 0 00000100:00100000:0.0:1789760781.763882:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.763990:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508453632 00000100:00100000:2.0:1789760781.763999:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508453632 00000100:00100000:2.0:1789760781.764051:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760791342418 time 760829894751 state 0x0 ref 1 00000100:00100000:2.0:1789760781.764067:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760830155167 time 760829894751 state 0x1 ref 1 00000100:00100000:2.0:1789760781.764096:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760830155167 time 760868807542 state 0x0 ref 1 00000100:00100000:2.0:1789760781.764109:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508453632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760781.764154:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24551424, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:2.0:1789760781.764163:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56af5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.764178:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.764182:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.764188:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56af5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.764206:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56af5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.800522:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973314 is committed 00002000:00010000:1.0:1789760781.801334:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24551424 -> 24555520 00010000:00010000:1.0:1789760781.801350:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56af5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.801363:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56af5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.801371:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56af5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.801377:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760781.801382:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56af5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.801390:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56af5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.801408:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56af5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.801415:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56af5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24551424->24555519] (req 24551424->24555519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.801520:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508453632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37407us (37642us total) trans 4294973314 rc 0/0 00000100:00100000:1.0:1789760781.801534:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6026 00000100:00100000:1.0:1789760781.801544:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760830155167 time 760868807542 state 0x2 ref 0 00000100:00100000:3.0:1789760781.804571:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760781.804644:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508453760 00000100:00100000:1.0:1789760781.804658:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508453760 00000100:00100000:1.0:1789760781.804720:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760830155167 time 760868807542 state 0x0 ref 1 00000100:00100000:1.0:1789760781.804737:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760869067958 time 760868807542 state 0x1 ref 1 00000100:00100000:1.0:1789760781.804769:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760869067958 time 760909479843 state 0x0 ref 1 00000100:00100000:1.0:1789760781.804793:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508453760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760781.804915:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24555520, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:1.0:1789760781.804925:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56afc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.804939:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760781.804943:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.804950:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56afc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.804964:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56afc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.846307:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973315 is committed 00002000:00010000:1.0:1789760781.846454:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24555520 -> 24559616 00010000:00010000:1.0:1789760781.846473:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56afc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.846490:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56afc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.846501:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56afc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.846509:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760781.846514:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56afc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.846524:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56afc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.846549:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56afc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.846559:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56afc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24555520->24559615] (req 24555520->24559615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.846679:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508453760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41880us (42113us total) trans 4294973315 rc 0/0 00000100:00100000:1.0:1789760781.846690:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6027 00000100:00100000:1.0:1789760781.846703:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760869067958 time 760909479843 state 0x2 ref 0 00000100:00100000:3.0:1789760781.850690:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760781.850813:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508453888 00000100:00100000:2.0:1789760781.850827:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508453888 00000100:00100000:2.0:1789760781.850900:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760869067958 time 760909479843 state 0x0 ref 1 00000100:00100000:2.0:1789760781.850920:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760909740259 time 760909479843 state 0x1 ref 1 00000100:00100000:2.0:1789760781.850959:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760909740259 time 760955668971 state 0x0 ref 1 00000100:00100000:2.0:1789760781.850974:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508453888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760781.851038:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24559616, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:2.0:1789760781.851049:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56b03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.851070:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:2.0:1789760781.851075:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.851082:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56b03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.851099:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56b03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.879799:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973316 is committed 00002000:00010000:1.0:1789760781.879906:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24559616 -> 24563712 00010000:00010000:1.0:1789760781.879926:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56b03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.879941:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56b03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.879950:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56b03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.879957:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760781.879963:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56b03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.879974:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56b03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.879996:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56b03 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.880003:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56b03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24559616->24563711] (req 24559616->24563711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.880116:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508453888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29139us (29435us total) trans 4294973316 rc 0/0 00000100:00100000:1.0:1789760781.880134:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6028 00000100:00100000:1.0:1789760781.880146:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760909740259 time 760955668971 state 0x2 ref 0 00000100:00100000:3.0:1789760781.887015:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760781.887121:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508454016 00000100:00100000:1.0:1789760781.887133:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508454016 00000100:00100000:1.0:1789760781.887202:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760909740259 time 760955668971 state 0x0 ref 1 00000100:00100000:1.0:1789760781.887220:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760955929387 time 760955668971 state 0x1 ref 1 00000100:00100000:1.0:1789760781.887282:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760955929387 time 760991992176 state 0x0 ref 1 00000100:00100000:1.0:1789760781.887349:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508454016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760781.887411:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24563712, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:1.0:1789760781.887422:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56b0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.887441:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=760s v=(0 0) 00010000:00010000:1.0:1789760781.887446:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.887454:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56b0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.887470:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56b0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760781.922028:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24563712 -> 24567808 00010000:00010000:2.0:1789760781.922055:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56b0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.922072:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56b0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.922082:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56b0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.922091:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760781.922096:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56b0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.922107:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56b0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.922148:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56b0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.922156:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56b0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24563712->24567807] (req 24563712->24567807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.922278:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508454016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34923us (35269us total) trans 4294973317 rc 0/0 00000100:00100000:2.0:1789760781.922333:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6029 00000100:00100000:2.0:1789760781.922493:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760955929387 time 760991992176 state 0x2 ref 0 00000001:00080000:2.0:1789760781.922547:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973317 is committed 00000100:00100000:3.0:1789760781.925033:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760781.925111:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508454144 00000100:00100000:1.0:1789760781.925124:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508454144 00000100:00100000:1.0:1789760781.925191:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760955929387 time 760991992176 state 0x0 ref 1 00000100:00100000:1.0:1789760781.925206:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760992252592 time 760991992176 state 0x1 ref 1 00000100:00100000:1.0:1789760781.925241:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760992252592 time 761029950876 state 0x0 ref 1 00000100:00100000:1.0:1789760781.925255:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508454144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760781.925366:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24567808, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:1.0:1789760781.925377:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56b11 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.925397:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760781.925402:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.925408:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56b11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.925423:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56b11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.960121:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973318 is committed 00002000:00010000:2.0:1789760781.960379:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24567808 -> 24571904 00010000:00010000:2.0:1789760781.960398:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56b11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.960414:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56b11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.960425:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56b11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.960434:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760781.960440:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56b11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.960451:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56b11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.960476:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56b11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.960485:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe56b11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24567808->24571903] (req 24567808->24571903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.960946:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508454144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35684us (35918us total) trans 4294973318 rc 0/0 00000100:00100000:3.0:1789760781.960960:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6030 00000100:00100000:3.0:1789760781.960973:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760992252592 time 761029950876 state 0x2 ref 0 00000100:00100000:3.0:1789760781.965438:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760781.965669:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508454272 00000100:00100000:1.0:1789760781.965685:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508454272 00000100:00100000:1.0:1789760781.966489:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 760992252592 time 761029950876 state 0x0 ref 1 00000100:00100000:1.0:1789760781.966511:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761030211292 time 761029950876 state 0x1 ref 1 00000100:00100000:1.0:1789760781.966549:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761030211292 time 761071259744 state 0x0 ref 1 00000100:00100000:1.0:1789760781.966568:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508454272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760781.966629:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24571904, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:1.0:1789760781.966640:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.966659:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760781.966664:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.966672:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.966688:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.999423:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973319 is committed 00002000:00010000:1.0:1789760781.999549:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24571904 -> 24576000 00010000:00010000:1.0:1789760781.999570:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.999586:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.999596:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.999603:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760781.999609:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.999619:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b18 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.999648:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b18 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.999656:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24571904->24575999] (req 24571904->24575999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.999969:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508454272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33336us (34485us total) trans 4294973319 rc 0/0 00000100:00100000:1.0:1789760781.999982:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6031 00000100:00100000:1.0:1789760781.999996:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761030211292 time 761071259744 state 0x2 ref 0 00000100:00100000:0.0:1789760782.003732:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760782.004198:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508454400 00000100:00100000:2.0:1789760782.004213:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508454400 00000100:00100000:2.0:1789760782.004454:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761030211292 time 761071259744 state 0x0 ref 1 00000100:00100000:2.0:1789760782.004474:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761071520160 time 761071259744 state 0x1 ref 1 00000100:00100000:2.0:1789760782.004554:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761071520160 time 761109263055 state 0x0 ref 1 00000100:00100000:2.0:1789760782.004569:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508454400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760782.004634:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24576000, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:2.0:1789760782.004644:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56b1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.004663:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760782.004669:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.004676:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56b1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.004704:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56b1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.047910:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973320 is committed 00002000:00010000:2.0:1789760782.048247:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24576000 -> 24580096 00010000:00010000:2.0:1789760782.048273:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56b1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.048330:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56b1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.048340:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56b1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.048348:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760782.048354:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56b1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.048365:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56b1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.048391:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56b1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.048399:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56b1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24576000->24580095] (req 24576000->24580095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.048523:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508454400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43950us (44798us total) trans 4294973320 rc 0/0 00000100:00100000:2.0:1789760782.048535:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6032 00000100:00100000:2.0:1789760782.048547:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761071520160 time 761109263055 state 0x2 ref 0 00000100:00100000:3.0:1789760782.052239:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760782.052391:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508454528 00000100:00100000:1.0:1789760782.052406:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508454528 00000100:00100000:1.0:1789760782.052481:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761071520160 time 761109263055 state 0x0 ref 1 00000100:00100000:1.0:1789760782.052500:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761109523471 time 761109263055 state 0x1 ref 1 00000100:00100000:1.0:1789760782.052631:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761109523471 time 761157340812 state 0x0 ref 1 00000100:00100000:1.0:1789760782.052645:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508454528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760782.052710:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24580096, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:1.0:1789760782.052721:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.052741:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760782.052748:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.052756:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.052783:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.079066:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760782.088251:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973321 is committed 00002000:00010000:3.0:1789760782.088422:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24580096 -> 24584192 00010000:00010000:3.0:1789760782.088443:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.088459:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.088469:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.088476:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760782.088481:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.088504:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.088670:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.088681:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56b26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24580096->24584191] (req 24580096->24584191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.088884:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508454528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36233us (36655us total) trans 4294973321 rc 0/0 00000100:00100000:3.0:1789760782.088896:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6033 00000100:00100000:3.0:1789760782.088910:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761109523471 time 761157340812 state 0x2 ref 0 00000100:00100000:3.0:1789760782.093731:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760782.094052:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508454656 00000100:00100000:1.0:1789760782.094067:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508454656 00000100:00100000:1.0:1789760782.094150:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761109523471 time 761157340812 state 0x0 ref 1 00000100:00100000:1.0:1789760782.094169:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761157601228 time 761157340812 state 0x1 ref 1 00000100:00100000:1.0:1789760782.094210:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761157601228 time 761198920105 state 0x0 ref 1 00000100:00100000:1.0:1789760782.094224:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508454656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760782.094307:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24584192, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:1.0:1789760782.094318:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56b2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.094339:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760782.094344:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.094353:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56b2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.094371:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56b2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.157487:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973322 is committed 00002000:00010000:1.0:1789760782.157583:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24584192 -> 24588288 00010000:00010000:1.0:1789760782.157603:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56b2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.157619:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56b2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.157628:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56b2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.157636:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760782.157641:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56b2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.157652:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56b2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.157675:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56b2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.157743:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56b2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24584192->24588287] (req 24584192->24588287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.157899:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508454656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 63669us (64176us total) trans 4294973322 rc 0/0 00000100:00100000:1.0:1789760782.157915:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6034 00000100:00100000:1.0:1789760782.157927:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761157601228 time 761198920105 state 0x2 ref 0 00000100:00100000:2.0:1789760782.161227:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760782.161319:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508454784 00000100:00100000:3.0:1789760782.161332:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508454784 00000100:00100000:3.0:1789760782.161405:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761157601228 time 761198920105 state 0x0 ref 1 00000100:00100000:3.0:1789760782.161424:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761199180521 time 761198920105 state 0x1 ref 1 00000100:00100000:3.0:1789760782.161460:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761199180521 time 761266170180 state 0x0 ref 1 00000100:00100000:3.0:1789760782.161473:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508454784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760782.161541:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24588288, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:3.0:1789760782.161552:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.161572:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760782.161577:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.161585:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.161601:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.205781:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973323 is committed 00002000:00010000:3.0:1789760782.205811:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24588288 -> 24592384 00010000:00010000:3.0:1789760782.205833:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.205850:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.205860:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.205868:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760782.205874:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.205885:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.205911:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.205919:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24588288->24592383] (req 24588288->24592383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.206098:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508454784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44614us (44874us total) trans 4294973323 rc 0/0 00000100:00100000:3.0:1789760782.206110:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6035 00000100:00100000:3.0:1789760782.206130:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761199180521 time 761266170180 state 0x2 ref 0 00000100:00100000:1.0:1789760782.206969:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7342300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534086912:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760782.207046:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760782.207190:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534086912 00000100:00100000:2.0:1789760782.207200:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534086912 00000100:00100000:3.0:1789760782.207240:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534087040:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760782.207267:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:3.0:1789760782.207271:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760782.207278:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534086912:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760782.207328:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3833593856 avail 3800039424 00000100:00100000:2.0:1789760782.207373:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534086912:12345-0@lo:13:osp-pre-0-1.0 Request processed in 96us (326us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.207380:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:2.0:1789760782.207396:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534087040 00000100:00100000:2.0:1789760782.207399:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534087040 00000100:00100000:2.0:1789760782.207420:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:2.0:1789760782.207426:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534087040:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760782.207437:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3833593856 avail 3800039424 00000100:00100000:3.0:1789760782.207437:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0498546b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534087168:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760782.207462:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760782.207463:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534087040:12345-0@lo:13:osp-pre-0-0.0 Request processed in 38us (191us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.207469:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:2.0:1789760782.207478:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534087168 00000100:00100000:2.0:1789760782.207482:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534087168 00000100:00100000:2.0:1789760782.207504:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:2.0:1789760782.207510:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534087168:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789760782.207537:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789760782.207569:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534087168:12345-0@lo:13:osp-pre-1-1.0 Request processed in 59us (106us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.207574:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00080000:2.0:1789760782.207593:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760782.207598:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760782.207738:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534087040:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760782.207820:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0498546b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534087168:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760782.207868:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7342300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534086912:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760782.208945:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789760782.208966:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7340000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3473:1876699534087424:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.208982:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7342300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534087680:0@lo:400:kworker.0 00000100:00100000:3.0:1789760782.209060:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7343800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534087808:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.210023:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.210136:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7341f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534087936:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.210221:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.210252:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760782.210252:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7340380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3475:1876699534088192:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.210273:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7341880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3473:1876699534088448:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.210282:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.210312:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760782.210327:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:2.0:1789760782.210334:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7340a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3475:1876699534087552:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.210360:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760782.210379:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7340e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3475:1876699534088064:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.210401:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.210413:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049beaf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534087296:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760782.210418:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7343b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534088576:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.210440:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.210441:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.210480:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534087680 00000100:00100000:0.0:1789760782.210490:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534087680 00000100:00100000:0.0:1789760782.210543:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534087936 00000100:00100000:0.0:1789760782.210547:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534087936 00000100:00100000:0.0:1789760782.210565:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:0.0:1789760782.210574:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534087680:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.210812:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534087680:12345-0@lo:400:kworker.0 Request processed in 236us (589us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.210820:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:0.0:1789760782.210842:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:0.0:1789760782.210848:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534087936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.210893:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534087936:12345-0@lo:400:kworker.0 Request processed in 45us (640us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.210898:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:0.0:1789760782.210916:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534088192 00000100:00100000:0.0:1789760782.210920:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534088192 00000100:00100000:1.0:1789760782.210976:0:6459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534087424 00000100:00100000:1.0:1789760782.210981:0:6459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534087424 00000100:00100000:1.0:1789760782.211014:0:6459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 488 00000100:00100000:1.0:1789760782.211022:0:6459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988caa050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3473:x1876699534087424:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.211057:0:6459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988caa050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3473:x1876699534087424:12345-0@lo:400:kworker.0 Request processed in 38us (1041us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.211064:0:6459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 488 00000100:00100000:0.0:1789760782.211609:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534088448 00000100:00100000:0.0:1789760782.211615:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534088448 00000100:00100000:1.0:1789760782.211705:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534087552 00000100:00100000:1.0:1789760782.211710:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534087552 00000100:00100000:1.0:1789760782.211736:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534088064 00000100:00100000:1.0:1789760782.211740:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534088064 00000100:00100000:1.0:1789760782.211760:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 973 00000100:00100000:1.0:1789760782.211782:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3475:x1876699534088192:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.211829:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3475:x1876699534088192:12345-0@lo:400:kworker.0 Request processed in 61us (1545us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.211834:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 973 00000100:00100000:1.0:1789760782.211849:0:11662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 974 00000100:00100000:1.0:1789760782.211854:0:11662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3475:x1876699534087552:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.211965:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 975 00000100:00100000:0.0:1789760782.211971:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3475:x1876699534088064:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.212007:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3475:x1876699534088064:12345-0@lo:400:kworker.0 Request processed in 36us (1605us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.212014:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 975 00000100:00100000:0.0:1789760782.212038:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 976 00000100:00100000:0.0:1789760782.212044:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3473:x1876699534088448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.212073:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3473:x1876699534088448:12345-0@lo:400:kworker.0 Request processed in 30us (1761us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.212079:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 976 00000100:00100000:0.0:1789760782.212134:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7340000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3473:1876699534087424:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.212152:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7341f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534087936:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.212323:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7342300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534087680:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.212470:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7341880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3473:1876699534088448:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.212583:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760782.212647:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508454912 00000100:00100000:3.0:1789760782.212654:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508454912 00000100:00100000:1.0:1789760782.212801:0:11662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3475:x1876699534087552:12345-0@lo:400:kworker.0 Request processed in 945us (2437us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.212809:0:11662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 974 00000100:00100000:2.0:1789760782.212981:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7340380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3475:1876699534088192:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.213006:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7340a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3475:1876699534087552:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.213017:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7340e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3475:1876699534088064:0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.213061:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534088576 00000100:00100000:2.0:1789760782.213066:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534088576 00000100:00100000:2.0:1789760782.213113:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 247 00000100:00100000:2.0:1789760782.213122:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534088576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.213236:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b7fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534088576:12345-0@lo:400:kworker.0 Request processed in 115us (2794us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.213246:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 247 00000100:00100000:0.0:1789760782.213248:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7343b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534088576:0@lo:400:kworker.0 00000100:00080000:2.0:1789760782.213264:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760782.213270:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760782.213273:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789760782.213311:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534087296 00000100:00100000:2.0:1789760782.213315:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534087296 00000100:00100000:2.0:1789760782.213411:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:2.0:1789760782.213418:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534087296:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760782.213472:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760782.213557:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534087296:12345-0@lo:13:osp-pre-1-0.0 Request processed in 138us (3113us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.213564:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1789760782.213680:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761199180521 time 761266170180 state 0x0 ref 1 00000100:00100000:3.0:1789760782.213698:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761266430596 time 761266170180 state 0x1 ref 1 00000100:00100000:3.0:1789760782.213727:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761266430596 time 761318437100 state 0x0 ref 1 00000100:00100000:3.0:1789760782.213739:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508454912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760782.213809:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24592384, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:3.0:1789760782.213821:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.213846:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760782.213859:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.213868:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.214247:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.214336:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049beaf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534087296:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760782.215450:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760782.215507:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7341500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3472:1876699534088320:0@lo:400:kworker.0 00000100:00100000:3.0:1789760782.215540:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760782.218390:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534087808 00000100:00100000:1.0:1789760782.218417:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534087808 00000100:00100000:0.0:1789760782.219570:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534088320 00000100:00100000:0.0:1789760782.219584:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534088320 00000100:00100000:2.0:1789760782.219756:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 977 00000100:00100000:2.0:1789760782.219780:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534087808:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.219897:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534087808:12345-0@lo:400:kworker.0 Request processed in 130us (4452us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.219905:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 977 00000100:00100000:2.0:1789760782.219926:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 978 00000100:00100000:2.0:1789760782.219933:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3472:x1876699534088320:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760782.219962:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3472:x1876699534088320:12345-0@lo:400:kworker.0 Request processed in 30us (4421us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.219968:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 978 00000100:00080000:3.0:1789760782.220052:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760782.220103:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7343800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534087808:0@lo:400:kworker.0 00000100:00100000:3.0:1789760782.220127:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7341500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3472:1876699534088320:0@lo:400:kworker.0 00000001:00080000:2.0:1789760782.260330:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973324 is committed 00002000:00010000:1.0:1789760782.260569:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24592384 -> 24596480 00010000:00010000:1.0:1789760782.260593:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.260609:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.260619:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.260628:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760782.260634:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.260645:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.260673:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.260686:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56b3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24592384->24596479] (req 24592384->24596479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.260830:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508454912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47087us (48243us total) trans 4294973324 rc 0/0 00000100:00100000:1.0:1789760782.260841:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6036 00000100:00100000:1.0:1789760782.260853:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761266430596 time 761318437100 state 0x2 ref 0 00000100:00100000:1.0:1789760782.264056:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760782.265482:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508455040 00000100:00100000:3.0:1789760782.265499:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508455040 00000100:00100000:3.0:1789760782.265583:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761266430596 time 761318437100 state 0x0 ref 1 00000100:00100000:3.0:1789760782.265602:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761318697516 time 761318437100 state 0x1 ref 1 00000100:00100000:3.0:1789760782.265636:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761318697516 time 761370346416 state 0x0 ref 1 00000100:00100000:3.0:1789760782.265650:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508455040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760782.265723:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24596480, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:3.0:1789760782.265734:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.265755:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760782.265760:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.265775:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.265794:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.311805:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973325 is committed 00002000:00010000:3.0:1789760782.311934:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24596480 -> 24600576 00010000:00010000:3.0:1789760782.311957:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.311973:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.311983:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.311991:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760782.311996:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.312008:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.312041:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.312050:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24596480->24600575] (req 24596480->24600575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.312283:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508455040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46626us (48231us total) trans 4294973325 rc 0/0 00000100:00100000:3.0:1789760782.312338:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6037 00000100:00100000:3.0:1789760782.312350:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761318697516 time 761370346416 state 0x2 ref 0 00000100:00100000:2.0:1789760782.323393:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760782.323548:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508455168 00000100:00100000:3.0:1789760782.323561:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508455168 00000100:00100000:3.0:1789760782.323636:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761318697516 time 761370346416 state 0x0 ref 1 00000100:00100000:3.0:1789760782.323655:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761370606832 time 761370346416 state 0x1 ref 1 00000100:00100000:3.0:1789760782.323691:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761370606832 time 761428402035 state 0x0 ref 1 00000100:00100000:3.0:1789760782.323705:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508455168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760782.323788:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24600576, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:3.0:1789760782.323801:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.323823:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760782.323830:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.323838:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.323857:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760782.355674:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24600576 -> 24604672 00000001:00080000:1.0:1789760782.355684:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973326 is committed 00010000:00010000:3.0:1789760782.355706:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.355723:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.355733:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.355741:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760782.355746:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.355755:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.355798:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.355807:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24600576->24604671] (req 24600576->24604671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.356141:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508455168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32429us (32754us total) trans 4294973326 rc 0/0 00000100:00100000:3.0:1789760782.356156:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6038 00000100:00100000:3.0:1789760782.356242:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761370606832 time 761428402035 state 0x2 ref 0 00000100:00100000:0.0:1789760782.364549:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760782.364903:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508455296 00000100:00100000:1.0:1789760782.364916:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508455296 00000100:00100000:1.0:1789760782.365002:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761370606832 time 761428402035 state 0x0 ref 1 00000100:00100000:1.0:1789760782.365023:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761428662451 time 761428402035 state 0x1 ref 1 00000100:00100000:1.0:1789760782.365060:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761428662451 time 761469770137 state 0x0 ref 1 00000100:00100000:1.0:1789760782.365082:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508455296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760782.365157:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24604672, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:1.0:1789760782.365168:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.365188:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760782.365192:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.365201:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.365217:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.403109:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973327 is committed 00002000:00010000:1.0:1789760782.403222:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24604672 -> 24608768 00010000:00010000:1.0:1789760782.403242:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.403257:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.403267:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.403276:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760782.403281:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.403463:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.403489:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.403497:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24604672->24608767] (req 24604672->24608767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.403727:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508455296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38649us (39186us total) trans 4294973327 rc 0/0 00000100:00100000:1.0:1789760782.403740:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6039 00000100:00100000:1.0:1789760782.403753:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761428662451 time 761469770137 state 0x2 ref 0 00000100:00100000:1.0:1789760782.408794:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760782.408926:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508455424 00000100:00100000:3.0:1789760782.408940:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508455424 00000100:00100000:3.0:1789760782.409014:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761428662451 time 761469770137 state 0x0 ref 1 00000100:00100000:3.0:1789760782.409035:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761470030553 time 761469770137 state 0x1 ref 1 00000100:00100000:3.0:1789760782.409066:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761470030553 time 761513776505 state 0x0 ref 1 00000100:00100000:3.0:1789760782.409080:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508455424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760782.409144:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24608768, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:3.0:1789760782.409155:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.409175:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760782.409181:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.409189:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.409205:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.449805:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973328 is committed 00002000:00010000:3.0:1789760782.449844:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24608768 -> 24612864 00010000:00010000:3.0:1789760782.449961:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.449978:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.449988:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.449997:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760782.450003:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.450015:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.450042:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.450052:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24608768->24612863] (req 24608768->24612863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.450189:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508455424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41104us (41403us total) trans 4294973328 rc 0/0 00000100:00100000:3.0:1789760782.450207:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6040 00000100:00100000:3.0:1789760782.450222:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761470030553 time 761513776505 state 0x2 ref 0 00000100:00100000:2.0:1789760782.454541:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760782.455079:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508455552 00000100:00100000:3.0:1789760782.455092:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508455552 00000100:00100000:3.0:1789760782.455543:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761470030553 time 761513776505 state 0x0 ref 1 00000100:00100000:3.0:1789760782.455564:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761514036921 time 761513776505 state 0x1 ref 1 00000100:00100000:3.0:1789760782.455590:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761514036921 time 761560300737 state 0x0 ref 1 00000100:00100000:3.0:1789760782.455605:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508455552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760782.455676:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24612864, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:3.0:1789760782.455696:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.455717:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760782.455723:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.455731:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.455747:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760782.498842:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24612864 -> 24616960 00010000:00010000:2.0:1789760782.498871:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.498888:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.498900:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.498910:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760782.498916:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.498926:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.498953:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.499104:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56b5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24612864->24616959] (req 24612864->24616959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.499419:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508455552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43810us (44886us total) trans 4294973329 rc 0/0 00000100:00100000:2.0:1789760782.499432:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6041 00000100:00100000:2.0:1789760782.499445:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761514036921 time 761560300737 state 0x2 ref 0 00000001:00080000:0.0:1789760782.499945:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973329 is committed 00000100:00100000:0.0:1789760782.504081:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760782.504192:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508455680 00000100:00100000:1.0:1789760782.504204:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508455680 00000100:00100000:1.0:1789760782.504519:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761514036921 time 761560300737 state 0x0 ref 1 00000100:00100000:1.0:1789760782.504539:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761560561153 time 761560300737 state 0x1 ref 1 00000100:00100000:1.0:1789760782.504608:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761560561153 time 761609317298 state 0x0 ref 1 00000100:00100000:1.0:1789760782.504622:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508455680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760782.504686:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24616960, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0xc007 00010000:00010000:1.0:1789760782.504698:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.504796:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760782.504802:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.504809:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.504827:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.546598:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973330 is committed 00002000:00010000:3.0:1789760782.546751:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24616960 -> 24621056 00002000:00010000:3.0:1789760782.546757:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760782 -> 1789760783 00002000:00010000:3.0:1789760782.546761:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760782 -> 1789760783 00010000:00010000:3.0:1789760782.546785:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.546802:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.546811:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.546819:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760782.546825:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.546835:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.546861:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.546868:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe56b65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24616960->24621055] (req 24616960->24621055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.547031:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508455680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42400us (42953us total) trans 4294973330 rc 0/0 00000100:00100000:3.0:1789760782.547044:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6042 00000100:00100000:3.0:1789760782.547056:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761560561153 time 761609317298 state 0x2 ref 0 00000100:00100000:3.0:1789760782.551753:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760782.552210:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508455808 00000100:00100000:1.0:1789760782.552224:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508455808 00000100:00100000:1.0:1789760782.552381:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761560561153 time 761609317298 state 0x0 ref 1 00000100:00100000:1.0:1789760782.552400:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761609577714 time 761609317298 state 0x1 ref 1 00000100:00100000:1.0:1789760782.552424:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761609577714 time 761657134737 state 0x0 ref 1 00000100:00100000:1.0:1789760782.552437:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508455808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760782.552500:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24621056, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:1.0:1789760782.552510:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b6c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.552528:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760782.552533:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.552540:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.552556:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760782.594053:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24621056 -> 24625152 00010000:00010000:1.0:1789760782.594080:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.594101:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.594110:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.594129:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760782.594135:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.594146:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.594170:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.594178:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24621056->24625151] (req 24621056->24625151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.594352:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508455808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41910us (42607us total) trans 4294973331 rc 0/0 00000100:00100000:1.0:1789760782.594365:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6043 00000100:00100000:1.0:1789760782.594378:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761609577714 time 761657134737 state 0x2 ref 0 00000001:00080000:3.0:1789760782.597265:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973331 is committed 00000100:00100000:0.0:1789760782.597594:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760782.597890:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508455936 00000100:00100000:2.0:1789760782.597904:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508455936 00000100:00100000:2.0:1789760782.597978:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761609577714 time 761657134737 state 0x0 ref 1 00000100:00100000:2.0:1789760782.597997:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761657395153 time 761657134737 state 0x1 ref 1 00000100:00100000:2.0:1789760782.598034:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761657395153 time 761702743464 state 0x0 ref 1 00000100:00100000:2.0:1789760782.598049:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508455936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760782.598112:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24625152, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:2.0:1789760782.598401:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.598420:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760782.598425:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.598433:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.598450:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.632566:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973332 is committed 00002000:00010000:2.0:1789760782.632675:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24625152 -> 24629248 00010000:00010000:2.0:1789760782.632775:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.632791:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.632802:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.632810:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760782.632816:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.632827:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.632855:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.632863:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24625152->24629247] (req 24625152->24629247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.633033:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508455936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34980us (35438us total) trans 4294973332 rc 0/0 00000100:00100000:2.0:1789760782.633046:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6044 00000100:00100000:2.0:1789760782.633059:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761657395153 time 761702743464 state 0x2 ref 0 00000100:00100000:3.0:1789760782.636439:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760782.636531:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508456064 00000100:00100000:1.0:1789760782.636543:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508456064 00000100:00100000:1.0:1789760782.636610:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761657395153 time 761702743464 state 0x0 ref 1 00000100:00100000:1.0:1789760782.636628:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761703003880 time 761702743464 state 0x1 ref 1 00000100:00100000:1.0:1789760782.636658:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761703003880 time 761741369141 state 0x0 ref 1 00000100:00100000:1.0:1789760782.636671:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508456064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760782.636741:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24629248, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:1.0:1789760782.636752:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.636780:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760782.636785:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.636793:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.636809:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.664041:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973333 is committed 00002000:00010000:3.0:1789760782.664242:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24629248 -> 24633344 00010000:00010000:3.0:1789760782.664262:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.664278:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.664381:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.664390:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760782.664396:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.664407:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.664432:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.664441:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56b7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24629248->24633343] (req 24629248->24633343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.664761:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508456064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28082us (28328us total) trans 4294973333 rc 0/0 00000100:00100000:2.0:1789760782.664782:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6045 00000100:00100000:2.0:1789760782.664795:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761703003880 time 761741369141 state 0x2 ref 0 00000100:00100000:2.0:1789760782.672360:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760782.672496:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508456192 00000100:00100000:1.0:1789760782.672508:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508456192 00000100:00100000:1.0:1789760782.672578:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761703003880 time 761741369141 state 0x0 ref 1 00000100:00100000:1.0:1789760782.672597:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761741629557 time 761741369141 state 0x1 ref 1 00000100:00100000:1.0:1789760782.672630:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761741629557 time 761777340246 state 0x0 ref 1 00000100:00100000:1.0:1789760782.672648:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508456192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760782.672845:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24633344, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:1.0:1789760782.672857:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56b81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.672877:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760782.672882:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.672891:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56b81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.672908:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56b81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760782.710087:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24633344 -> 24637440 00010000:00010000:1.0:1789760782.710115:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56b81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.710132:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56b81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.710142:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56b81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.710151:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760782.710157:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56b81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.710167:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.710194:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56b81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.710201:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56b81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24633344->24637439] (req 24633344->24637439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.710373:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508456192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37719us (38021us total) trans 4294973334 rc 0/0 00000100:00100000:1.0:1789760782.710386:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6046 00000100:00100000:1.0:1789760782.710401:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761741629557 time 761777340246 state 0x2 ref 0 00000001:00080000:3.0:1789760782.710806:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973334 is committed 00000100:00100000:0.0:1789760782.717705:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760782.717829:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508456320 00000100:00100000:2.0:1789760782.717843:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508456320 00000100:00100000:2.0:1789760782.717919:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761741629557 time 761777340246 state 0x0 ref 1 00000100:00100000:2.0:1789760782.717939:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761777600662 time 761777340246 state 0x1 ref 1 00000100:00100000:2.0:1789760782.717968:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761777600662 time 761822678346 state 0x0 ref 1 00000100:00100000:2.0:1789760782.717983:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508456320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760782.718046:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24637440, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:2.0:1789760782.718056:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.718076:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760782.718081:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.718088:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.718104:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760782.749506:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24637440 -> 24641536 00010000:00010000:3.0:1789760782.749534:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.749551:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.749562:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.749571:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760782.749577:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.749588:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.749615:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.749624:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56b88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24637440->24641535] (req 24637440->24641535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.749750:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508456320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31761us (32054us total) trans 4294973335 rc 0/0 00000100:00100000:3.0:1789760782.749778:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6047 00000100:00100000:3.0:1789760782.749791:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761777600662 time 761822678346 state 0x2 ref 0 00000001:00080000:1.0:1789760782.750326:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973335 is committed 00000100:00100000:1.0:1789760782.756478:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760782.756629:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508456448 00000100:00100000:2.0:1789760782.756643:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508456448 00000100:00100000:2.0:1789760782.756715:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761777600662 time 761822678346 state 0x0 ref 1 00000100:00100000:2.0:1789760782.756733:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761822938762 time 761822678346 state 0x1 ref 1 00000100:00100000:2.0:1789760782.756840:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761822938762 time 761861548651 state 0x0 ref 1 00000100:00100000:2.0:1789760782.756867:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508456448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760782.756938:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24641536, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:2.0:1789760782.756948:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56b8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.756968:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:2.0:1789760782.756973:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.756981:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56b8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.756998:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56b8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.808182:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973336 is committed 00002000:00010000:2.0:1789760782.808391:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24641536 -> 24645632 00010000:00010000:2.0:1789760782.808408:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56b8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.808422:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56b8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.808432:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56b8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.808440:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760782.808444:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56b8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.808453:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56b8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.808473:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56b8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.808481:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56b8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24641536->24645631] (req 24641536->24645631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.808576:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508456448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51719us (52109us total) trans 4294973336 rc 0/0 00000100:00100000:2.0:1789760782.808587:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6048 00000100:00100000:2.0:1789760782.808597:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761822938762 time 761861548651 state 0x2 ref 0 00000100:00100000:2.0:1789760782.813167:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760782.813248:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508456576 00000100:00100000:1.0:1789760782.813258:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508456576 00000100:00100000:1.0:1789760782.813345:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761822938762 time 761861548651 state 0x0 ref 1 00000100:00100000:1.0:1789760782.813361:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761861809067 time 761861548651 state 0x1 ref 1 00000100:00100000:1.0:1789760782.813381:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761861809067 time 761918091592 state 0x0 ref 1 00000100:00100000:1.0:1789760782.813392:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508456576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760782.813436:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24645632, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:1.0:1789760782.813445:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56b96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.813461:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:1.0:1789760782.813465:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.813471:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56b96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.813485:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56b96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.844825:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973337 is committed 00002000:00010000:1.0:1789760782.845049:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24645632 -> 24649728 00010000:00010000:1.0:1789760782.845067:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56b96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.845081:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56b96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.845091:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56b96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.845098:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760782.845103:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56b96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.845176:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56b96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.845201:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56b96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.845209:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56b96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24645632->24649727] (req 24645632->24649727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.845378:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508456576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31983us (32217us total) trans 4294973337 rc 0/0 00000100:00100000:1.0:1789760782.845389:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6049 00000100:00100000:1.0:1789760782.845399:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761861809067 time 761918091592 state 0x2 ref 0 00000100:00100000:1.0:1789760782.850490:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760782.850608:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508456704 00000100:00100000:3.0:1789760782.850617:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508456704 00000100:00100000:3.0:1789760782.850671:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761861809067 time 761918091592 state 0x0 ref 1 00000100:00100000:3.0:1789760782.850686:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761918352008 time 761918091592 state 0x1 ref 1 00000100:00100000:3.0:1789760782.850719:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761918352008 time 761955429345 state 0x0 ref 1 00000100:00100000:3.0:1789760782.850732:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508456704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760782.850787:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24649728, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:3.0:1789760782.850797:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56b9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.850813:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=761s v=(0 0) 00010000:00010000:3.0:1789760782.850825:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.850832:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56b9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.850847:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56b9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.887093:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973338 is committed 00002000:00010000:2.0:1789760782.887425:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24649728 -> 24653824 00010000:00010000:2.0:1789760782.887439:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56b9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.887451:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56b9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.887459:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56b9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.887466:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760782.887471:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56b9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.887481:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56b9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.887500:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56b9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.887507:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56b9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24649728->24653823] (req 24649728->24653823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.887680:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508456704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36946us (37196us total) trans 4294973338 rc 0/0 00000100:00100000:2.0:1789760782.887691:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6050 00000100:00100000:2.0:1789760782.887701:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761918352008 time 761955429345 state 0x2 ref 0 00000100:00100000:3.0:1789760782.902469:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760782.905213:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508456832 00000100:00100000:2.0:1789760782.905228:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508456832 00000100:00100000:2.0:1789760782.905311:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761918352008 time 761955429345 state 0x0 ref 1 00000100:00100000:2.0:1789760782.905326:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761955689761 time 761955429345 state 0x1 ref 1 00000100:00100000:2.0:1789760782.905411:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761955689761 time 762010120084 state 0x0 ref 1 00000100:00100000:2.0:1789760782.905427:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508456832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760782.905475:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24653824, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:2.0:1789760782.905484:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56ba4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.905500:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760782.905505:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.905511:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56ba4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.905528:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56ba4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.935214:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973339 is committed 00002000:00010000:0.0:1789760782.938778:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24653824 -> 24657920 00010000:00010000:0.0:1789760782.938798:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56ba4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.938812:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56ba4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.938821:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56ba4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.938828:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760782.938833:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56ba4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.938842:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56ba4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.938864:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56ba4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.938872:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56ba4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24653824->24657919] (req 24653824->24657919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.939026:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508456832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33598us (36561us total) trans 4294973339 rc 0/0 00000100:00100000:0.0:1789760782.939038:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6051 00000100:00100000:0.0:1789760782.939047:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761955689761 time 762010120084 state 0x2 ref 0 00000100:00100000:0.0:1789760782.942740:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760782.943128:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508456960 00000100:00100000:1.0:1789760782.943138:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508456960 00000100:00100000:1.0:1789760782.943191:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 761955689761 time 762010120084 state 0x0 ref 1 00000100:00100000:1.0:1789760782.943206:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762010380500 time 762010120084 state 0x1 ref 1 00000100:00100000:1.0:1789760782.943239:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762010380500 time 762047948690 state 0x0 ref 1 00000100:00100000:1.0:1789760782.943251:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508456960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760782.943339:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24657920, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:1.0:1789760782.943348:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.943369:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760782.943373:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.943380:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.943393:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.985467:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973340 is committed 00002000:00010000:1.0:1789760782.985579:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24657920 -> 24662016 00010000:00010000:1.0:1789760782.985595:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.985609:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.985618:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.985625:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760782.985630:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.985640:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.985659:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.985668:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24657920->24662015] (req 24657920->24662015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.985912:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508456960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42658us (43176us total) trans 4294973340 rc 0/0 00000100:00100000:1.0:1789760782.985923:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6052 00000100:00100000:1.0:1789760782.985933:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762010380500 time 762047948690 state 0x2 ref 0 00000100:00100000:1.0:1789760782.991441:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760782.991812:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508457088 00000100:00100000:2.0:1789760782.991821:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508457088 00000100:00100000:2.0:1789760782.991873:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762010380500 time 762047948690 state 0x0 ref 1 00000100:00100000:2.0:1789760782.991888:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762048209106 time 762047948690 state 0x1 ref 1 00000100:00100000:2.0:1789760782.991974:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762048209106 time 762096683979 state 0x0 ref 1 00000100:00100000:2.0:1789760782.991986:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508457088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760782.992031:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24662016, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:2.0:1789760782.992040:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56bb2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.992055:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760782.992060:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.992066:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56bb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.992081:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56bb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760783.022952:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24662016 -> 24666112 00010000:00010000:0.0:1789760783.022971:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56bb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.022985:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56bb2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.022994:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56bb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.023000:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760783.023005:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56bb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.023013:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56bb2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.023031:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56bb2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.023038:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56bb2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24662016->24666111] (req 24662016->24666111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.023137:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508457088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31148us (31701us total) trans 4294973341 rc 0/0 00000100:00100000:0.0:1789760783.023149:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6053 00000100:00100000:0.0:1789760783.023159:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762048209106 time 762096683979 state 0x2 ref 0 00000001:00080000:1.0:1789760783.023939:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973341 is committed 00000100:00100000:3.0:1789760783.027250:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760783.027393:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508457216 00000100:00100000:0.0:1789760783.027403:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508457216 00000100:00100000:0.0:1789760783.027472:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762048209106 time 762096683979 state 0x0 ref 1 00000100:00100000:0.0:1789760783.027487:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762096944395 time 762096683979 state 0x1 ref 1 00000100:00100000:0.0:1789760783.027524:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762096944395 time 762132227761 state 0x0 ref 1 00000100:00100000:0.0:1789760783.027538:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508457216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760783.027585:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24666112, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:0.0:1789760783.027594:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bb9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.027609:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.027614:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.027620:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.027644:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760783.032141:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760783.070317:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973342 is committed 00002000:00010000:1.0:1789760783.070434:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24666112 -> 24670208 00010000:00010000:1.0:1789760783.070450:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.070463:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.070472:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.070479:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760783.070483:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.070491:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.070512:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bb9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.070519:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bb9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24666112->24670207] (req 24666112->24670207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.070621:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508457216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43080us (43376us total) trans 4294973342 rc 0/0 00000100:00100000:1.0:1789760783.070630:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6054 00000100:00100000:1.0:1789760783.070639:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762096944395 time 762132227761 state 0x2 ref 0 00000100:00100000:3.0:1789760783.073602:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760783.073716:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508457344 00000100:00100000:1.0:1789760783.073726:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508457344 00000100:00100000:1.0:1789760783.073790:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762096944395 time 762132227761 state 0x0 ref 1 00000100:00100000:1.0:1789760783.073804:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762132488177 time 762132227761 state 0x1 ref 1 00000100:00100000:1.0:1789760783.073831:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762132488177 time 762178542200 state 0x0 ref 1 00000100:00100000:1.0:1789760783.073842:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508457344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760783.073892:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24670208, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:1.0:1789760783.073902:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bc0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.073917:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760783.073922:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.073928:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.073942:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.105082:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760783.126949:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973343 is committed 00002000:00010000:3.0:1789760783.127061:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24670208 -> 24674304 00010000:00010000:3.0:1789760783.127081:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.127094:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bc0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.127104:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.127241:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760783.127247:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.127257:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bc0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.127277:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bc0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.127303:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56bc0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24670208->24674303] (req 24670208->24674303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.127438:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508457344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53593us (53841us total) trans 4294973343 rc 0/0 00000100:00100000:3.0:1789760783.127452:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6055 00000100:00100000:3.0:1789760783.127461:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762132488177 time 762178542200 state 0x2 ref 0 00000100:00100000:2.0:1789760783.136842:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760783.137061:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508457472 00000100:00100000:0.0:1789760783.137070:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508457472 00000100:00100000:0.0:1789760783.137125:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762132488177 time 762178542200 state 0x0 ref 1 00000100:00100000:0.0:1789760783.137209:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762178802616 time 762178542200 state 0x1 ref 1 00000100:00100000:0.0:1789760783.137278:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762178802616 time 762241986901 state 0x0 ref 1 00000100:00100000:0.0:1789760783.137330:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508457472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760783.137381:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24674304, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:0.0:1789760783.137390:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bc7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.137405:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.137409:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.137416:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.137430:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.194053:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973344 is committed 00002000:00010000:3.0:1789760783.194163:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24674304 -> 24678400 00010000:00010000:3.0:1789760783.194177:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.194191:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bc7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.194201:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.194208:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760783.194212:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.194220:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bc7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.194239:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bc7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.194246:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56bc7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24674304->24678399] (req 24674304->24678399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.194376:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508457472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57044us (57538us total) trans 4294973344 rc 0/0 00000100:00100000:3.0:1789760783.194385:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6056 00000100:00100000:3.0:1789760783.194394:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762178802616 time 762241986901 state 0x2 ref 0 00000100:00100000:1.0:1789760783.199189:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760783.199277:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508457600 00000100:00100000:0.0:1789760783.199313:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508457600 00000100:00100000:0.0:1789760783.199364:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762178802616 time 762241986901 state 0x0 ref 1 00000100:00100000:0.0:1789760783.199380:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762242247317 time 762241986901 state 0x1 ref 1 00000100:00100000:0.0:1789760783.199415:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762242247317 time 762304125450 state 0x0 ref 1 00000100:00100000:0.0:1789760783.199427:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508457600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760783.199472:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24678400, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:0.0:1789760783.199480:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56bce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.199494:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.199498:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.199503:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56bce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.199518:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56bce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.231216:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888044380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534088704:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760783.231282:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760783.232207:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534088704 00000100:00100000:1.0:1789760783.232217:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534088704 00000100:00100000:1.0:1789760783.232273:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:1.0:1789760783.232283:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7343b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534088704:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760783.232342:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760783.232398:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7343b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534088704:12345-0@lo:41:osp-pre-0-1.0 Request processed in 116us (1115us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760783.232406:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00100000:1.0:1789760783.232456:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888044380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534088704:0@lo:41:osp-pre-0-1.0 00000001:00080000:0.0:1789760783.240965:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973345 is committed 00002000:00010000:3.0:1789760783.241149:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24678400 -> 24682496 00010000:00010000:3.0:1789760783.241166:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56bce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.241178:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56bce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.241188:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56bce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.241195:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760783.241200:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56bce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.241214:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56bce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.241233:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56bce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.241246:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56bce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24678400->24682495] (req 24678400->24682495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.241505:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508457600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42073us (42320us total) trans 4294973345 rc 0/0 00000100:00100000:3.0:1789760783.241515:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6057 00000100:00100000:3.0:1789760783.241524:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762242247317 time 762304125450 state 0x2 ref 0 00000100:00100000:1.0:1789760783.244888:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760783.245077:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508457728 00000100:00100000:3.0:1789760783.245086:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508457728 00000100:00100000:3.0:1789760783.245139:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762242247317 time 762304125450 state 0x0 ref 1 00000100:00100000:3.0:1789760783.245153:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762304385866 time 762304125450 state 0x1 ref 1 00000100:00100000:3.0:1789760783.245234:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762304385866 time 762349943129 state 0x0 ref 1 00000100:00100000:3.0:1789760783.245256:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508457728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760783.245325:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24682496, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:3.0:1789760783.245334:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56bd5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.245349:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760783.245354:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.245360:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56bd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.245379:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56bd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.283379:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973346 is committed 00002000:00010000:1.0:1789760783.283875:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24682496 -> 24686592 00010000:00010000:1.0:1789760783.283897:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56bd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.283910:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56bd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.283920:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56bd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.283932:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760783.283936:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56bd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.283945:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56bd5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.283965:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56bd5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.283972:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56bd5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24682496->24686591] (req 24682496->24686591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.284109:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508457728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38850us (39223us total) trans 4294973346 rc 0/0 00000100:00100000:1.0:1789760783.284126:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6058 00000100:00100000:1.0:1789760783.284135:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762304385866 time 762349943129 state 0x2 ref 0 00000100:00100000:1.0:1789760783.289819:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760783.289905:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508457856 00000100:00100000:3.0:1789760783.289914:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508457856 00000100:00100000:3.0:1789760783.289975:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762304385866 time 762349943129 state 0x0 ref 1 00000100:00100000:3.0:1789760783.289991:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762350203545 time 762349943129 state 0x1 ref 1 00000100:00100000:3.0:1789760783.290024:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762350203545 time 762394734980 state 0x0 ref 1 00000100:00100000:3.0:1789760783.290036:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508457856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760783.290086:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24686592, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:3.0:1789760783.290094:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56bdc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.290110:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760783.290114:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.290121:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56bdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.290134:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56bdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.324020:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973347 is committed 00002000:00010000:0.0:1789760783.324199:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24686592 -> 24690688 00010000:00010000:0.0:1789760783.324214:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56bdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.324225:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56bdc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.324238:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56bdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.324246:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760783.324251:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56bdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.324260:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56bdc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.324279:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56bdc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.324308:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56bdc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24686592->24690687] (req 24686592->24690687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.324402:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508457856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34363us (34588us total) trans 4294973347 rc 0/0 00000100:00100000:0.0:1789760783.324412:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6059 00000100:00100000:0.0:1789760783.324421:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762350203545 time 762394734980 state 0x2 ref 0 00000100:00100000:3.0:1789760783.331934:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760783.332489:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508457984 00000100:00100000:1.0:1789760783.332499:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508457984 00000100:00100000:1.0:1789760783.332557:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762350203545 time 762394734980 state 0x0 ref 1 00000100:00100000:1.0:1789760783.332572:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762394995396 time 762394734980 state 0x1 ref 1 00000100:00100000:1.0:1789760783.332606:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762394995396 time 762437316622 state 0x0 ref 1 00000100:00100000:1.0:1789760783.332618:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508457984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760783.332664:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24690688, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:1.0:1789760783.332680:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56be3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.332697:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:1.0:1789760783.332701:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.332707:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56be3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.332720:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56be3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.355370:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973348 is committed 00002000:00010000:3.0:1789760783.355469:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24690688 -> 24694784 00010000:00010000:3.0:1789760783.355486:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56be3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.355500:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56be3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.355509:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56be3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.355517:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760783.355522:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56be3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.355532:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56be3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.355552:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56be3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.355559:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe56be3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24690688->24694783] (req 24690688->24694783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.355650:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508457984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23029us (23721us total) trans 4294973348 rc 0/0 00000100:00100000:3.0:1789760783.355661:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6060 00000100:00100000:3.0:1789760783.355671:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762394995396 time 762437316622 state 0x2 ref 0 00000100:00100000:1.0:1789760783.359210:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760783.359494:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508458112 00000100:00100000:3.0:1789760783.359504:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508458112 00000100:00100000:3.0:1789760783.359555:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762394995396 time 762437316622 state 0x0 ref 1 00000100:00100000:3.0:1789760783.359571:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762437577038 time 762437316622 state 0x1 ref 1 00000100:00100000:3.0:1789760783.359600:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762437577038 time 762464311142 state 0x0 ref 1 00000100:00100000:3.0:1789760783.359613:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508458112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760783.359658:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24694784, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:3.0:1789760783.359666:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56bea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.359689:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760783.359693:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.359700:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56bea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.359714:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56bea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.400464:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973349 is committed 00002000:00010000:2.0:1789760783.400612:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24694784 -> 24698880 00010000:00010000:2.0:1789760783.400626:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56bea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.400640:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56bea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.400649:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56bea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.400657:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760783.400662:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56bea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.400671:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56bea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.400777:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56bea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.400791:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56bea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24694784->24698879] (req 24694784->24698879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.400883:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508458112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41267us (41679us total) trans 4294973349 rc 0/0 00000100:00100000:2.0:1789760783.400893:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6061 00000100:00100000:2.0:1789760783.400903:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762437577038 time 762464311142 state 0x2 ref 0 00000100:00100000:2.0:1789760783.404328:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760783.404424:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508458240 00000100:00100000:0.0:1789760783.404434:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508458240 00000100:00100000:0.0:1789760783.404486:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762437577038 time 762464311142 state 0x0 ref 1 00000100:00100000:0.0:1789760783.404504:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762464571558 time 762464311142 state 0x1 ref 1 00000100:00100000:0.0:1789760783.404563:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762464571558 time 762509272541 state 0x0 ref 1 00000100:00100000:0.0:1789760783.404575:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508458240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760783.404621:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24698880, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:0.0:1789760783.404630:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56bf1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.404645:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.404650:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.404656:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56bf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.404670:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56bf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.435626:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973350 is committed 00002000:00010000:0.0:1789760783.435741:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24698880 -> 24702976 00010000:00010000:0.0:1789760783.435767:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56bf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.435780:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56bf1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.435790:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56bf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.435803:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760783.435808:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56bf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.435817:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56bf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.435836:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56bf1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.435844:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56bf1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24698880->24702975] (req 24698880->24702975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.435975:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508458240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31396us (31650us total) trans 4294973350 rc 0/0 00000100:00100000:0.0:1789760783.435986:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6062 00000100:00100000:0.0:1789760783.435997:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762464571558 time 762509272541 state 0x2 ref 0 00000100:00100000:1.0:1789760783.440168:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760783.440258:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508458368 00000100:00100000:3.0:1789760783.440267:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508458368 00000100:00100000:3.0:1789760783.440498:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762464571558 time 762509272541 state 0x0 ref 1 00000100:00100000:3.0:1789760783.440632:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762509532957 time 762509272541 state 0x1 ref 1 00000100:00100000:3.0:1789760783.440756:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762509532957 time 762545464553 state 0x0 ref 1 00000100:00100000:3.0:1789760783.440781:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508458368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760783.440826:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24702976, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:3.0:1789760783.440836:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56bf8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.440852:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760783.440856:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.440863:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56bf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.440876:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56bf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.467982:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973351 is committed 00002000:00010000:1.0:1789760783.468156:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24702976 -> 24707072 00010000:00010000:1.0:1789760783.468172:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56bf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.468184:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56bf8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.468194:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56bf8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.468201:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760783.468206:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56bf8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.468215:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56bf8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.468234:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56bf8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.468241:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56bf8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24702976->24707071] (req 24702976->24707071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.468538:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508458368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27755us (28429us total) trans 4294973351 rc 0/0 00000100:00100000:0.0:1789760783.468549:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6063 00000100:00100000:0.0:1789760783.468558:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762509532957 time 762545464553 state 0x2 ref 0 00000100:00100000:0.0:1789760783.471979:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760783.472073:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508458496 00000100:00100000:3.0:1789760783.472084:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508458496 00000100:00100000:3.0:1789760783.472146:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762509532957 time 762545464553 state 0x0 ref 1 00000100:00100000:3.0:1789760783.472162:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762545724969 time 762545464553 state 0x1 ref 1 00000100:00100000:3.0:1789760783.472194:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762545724969 time 762576904959 state 0x0 ref 1 00000100:00100000:3.0:1789760783.472207:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508458496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760783.472255:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24707072, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0xc007 00010000:00010000:3.0:1789760783.472263:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56bff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.472279:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760783.472313:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.472468:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56bff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.472484:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56bff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.516322:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973352 is committed 00002000:00010000:3.0:1789760783.516433:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24707072 -> 24711168 00002000:00010000:3.0:1789760783.516438:0:14638:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760783 -> 1789760784 00002000:00010000:3.0:1789760783.516441:0:14638:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760783 -> 1789760784 00010000:00010000:3.0:1789760783.516454:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56bff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.516468:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56bff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.516477:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56bff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.516484:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760783.516489:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56bff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.516503:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56bff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.516523:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56bff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.516531:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56bff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24707072->24711167] (req 24707072->24711167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.516623:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508458496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44415us (44650us total) trans 4294973352 rc 0/0 00000100:00100000:3.0:1789760783.516634:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6064 00000100:00100000:3.0:1789760783.516644:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762545724969 time 762576904959 state 0x2 ref 0 00000100:00100000:1.0:1789760783.519635:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760783.519782:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508458624 00000100:00100000:3.0:1789760783.519792:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508458624 00000100:00100000:3.0:1789760783.519843:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762545724969 time 762576904959 state 0x0 ref 1 00000100:00100000:3.0:1789760783.519859:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762577165375 time 762576904959 state 0x1 ref 1 00000100:00100000:3.0:1789760783.519894:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762577165375 time 762624604074 state 0x0 ref 1 00000100:00100000:3.0:1789760783.519906:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508458624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760783.519954:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24711168, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:3.0:1789760783.519964:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.519979:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760783.519983:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.519989:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.520002:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760783.550558:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24711168 -> 24715264 00010000:00010000:0.0:1789760783.550580:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.550593:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.550602:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.550610:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760783.550615:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.550624:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c06 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.550643:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c06 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.550651:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24711168->24715263] (req 24711168->24715263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.550765:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508458624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30849us (31128us total) trans 4294973353 rc 0/0 00000100:00100000:0.0:1789760783.550777:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6065 00000100:00100000:0.0:1789760783.550786:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762577165375 time 762624604074 state 0x2 ref 0 00000001:00080000:0.0:1789760783.551729:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973353 is committed 00000100:00100000:0.0:1789760783.554080:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760783.554563:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508458752 00000100:00100000:3.0:1789760783.554573:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508458752 00000100:00100000:3.0:1789760783.554627:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762577165375 time 762624604074 state 0x0 ref 1 00000100:00100000:3.0:1789760783.554642:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762624864490 time 762624604074 state 0x1 ref 1 00000100:00100000:3.0:1789760783.554682:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762624864490 time 762659392283 state 0x0 ref 1 00000100:00100000:3.0:1789760783.554694:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508458752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760783.554741:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24715264, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:3.0:1789760783.554749:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.554780:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760783.554785:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.554792:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.554810:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760783.584093:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24715264 -> 24719360 00010000:00010000:3.0:1789760783.584128:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.584142:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.584151:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.584159:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760783.584164:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.584173:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.584194:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c0d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.584201:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24715264->24719359] (req 24715264->24719359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.584365:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508458752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29667us (30287us total) trans 4294973354 rc 0/0 00000100:00100000:3.0:1789760783.584376:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6066 00000100:00100000:3.0:1789760783.584386:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762624864490 time 762659392283 state 0x2 ref 0 00000001:00080000:0.0:1789760783.584610:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973354 is committed 00000100:00100000:2.0:1789760783.587919:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760783.588006:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508458880 00000100:00100000:3.0:1789760783.588015:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508458880 00000100:00100000:3.0:1789760783.588071:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762624864490 time 762659392283 state 0x0 ref 1 00000100:00100000:3.0:1789760783.588087:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762659652699 time 762659392283 state 0x1 ref 1 00000100:00100000:3.0:1789760783.588219:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762659652699 time 762692928719 state 0x0 ref 1 00000100:00100000:3.0:1789760783.588232:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508458880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760783.588280:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24719360, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:3.0:1789760783.588319:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.588334:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760783.588339:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.588345:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.588358:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.614360:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973355 is committed 00002000:00010000:3.0:1789760783.614502:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24719360 -> 24723456 00010000:00010000:3.0:1789760783.614519:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.614533:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.614542:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.614550:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760783.614554:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.614563:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c14 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.614582:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c14 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.614590:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24719360->24723455] (req 24719360->24723455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.614726:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508458880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26491us (26812us total) trans 4294973355 rc 0/0 00000100:00100000:3.0:1789760783.614740:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6067 00000100:00100000:3.0:1789760783.614750:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762659652699 time 762692928719 state 0x2 ref 0 00000100:00100000:2.0:1789760783.618354:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760783.618901:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508459008 00000100:00100000:0.0:1789760783.618912:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508459008 00000100:00100000:0.0:1789760783.618966:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762659652699 time 762692928719 state 0x0 ref 1 00000100:00100000:0.0:1789760783.618982:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762693189135 time 762692928719 state 0x1 ref 1 00000100:00100000:0.0:1789760783.619014:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762693189135 time 762723724765 state 0x0 ref 1 00000100:00100000:0.0:1789760783.619026:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508459008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760783.619073:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24723456, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:0.0:1789760783.619082:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56c1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.619099:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.619103:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.619120:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56c1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.619136:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56c1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.657872:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973356 is committed 00002000:00010000:0.0:1789760783.657985:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24723456 -> 24727552 00010000:00010000:0.0:1789760783.658000:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56c1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.658013:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56c1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.658022:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56c1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.658029:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760783.658033:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56c1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.658042:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56c1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.658064:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56c1b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.658072:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56c1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24723456->24727551] (req 24723456->24727551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.658183:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508459008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39154us (39833us total) trans 4294973356 rc 0/0 00000100:00100000:0.0:1789760783.658194:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6068 00000100:00100000:0.0:1789760783.658204:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762693189135 time 762723724765 state 0x2 ref 0 00000100:00100000:0.0:1789760783.661478:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760783.661596:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508459136 00000100:00100000:3.0:1789760783.661606:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508459136 00000100:00100000:3.0:1789760783.661658:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762693189135 time 762723724765 state 0x0 ref 1 00000100:00100000:3.0:1789760783.661684:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762723985181 time 762723724765 state 0x1 ref 1 00000100:00100000:3.0:1789760783.661714:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762723985181 time 762766424473 state 0x0 ref 1 00000100:00100000:3.0:1789760783.661726:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508459136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760783.661783:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24727552, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:3.0:1789760783.661792:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c22 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.661807:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:3.0:1789760783.661811:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.661818:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.661831:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.689125:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973357 is committed 00002000:00010000:2.0:1789760783.689454:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24727552 -> 24731648 00010000:00010000:2.0:1789760783.689469:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.689483:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c22 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.689492:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.689500:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760783.689505:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.689514:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c22 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.689535:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c22 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.689543:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe56c22 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24727552->24731647] (req 24727552->24731647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.689638:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508459136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27908us (28163us total) trans 4294973357 rc 0/0 00000100:00100000:2.0:1789760783.689649:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6069 00000100:00100000:2.0:1789760783.689659:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762723985181 time 762766424473 state 0x2 ref 0 00000100:00100000:3.0:1789760783.693544:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760783.693633:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508459264 00000100:00100000:2.0:1789760783.693642:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508459264 00000100:00100000:2.0:1789760783.693771:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762723985181 time 762766424473 state 0x0 ref 1 00000100:00100000:2.0:1789760783.693786:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762766684889 time 762766424473 state 0x1 ref 1 00000100:00100000:2.0:1789760783.693856:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762766684889 time 762798565310 state 0x0 ref 1 00000100:00100000:2.0:1789760783.693869:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d42050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508459264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760783.693921:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24731648, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:2.0:1789760783.693930:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.693945:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760783.693949:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.693955:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.693968:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760783.731965:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24731648 -> 24735744 00010000:00010000:2.0:1789760783.731988:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.732000:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.732009:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.732016:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760783.732020:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.732028:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.732054:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.732061:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24731648->24735743] (req 24731648->24735743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.732168:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d42050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508459264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38295us (38628us total) trans 4294973358 rc 0/0 00000100:00100000:2.0:1789760783.732178:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6070 00000100:00100000:2.0:1789760783.732187:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762766684889 time 762798565310 state 0x2 ref 0 00000001:00080000:3.0:1789760783.732530:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973358 is committed 00000100:00100000:3.0:1789760783.736145:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760783.736329:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508459392 00000100:00100000:0.0:1789760783.736339:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508459392 00000100:00100000:0.0:1789760783.736392:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762766684889 time 762798565310 state 0x0 ref 1 00000100:00100000:0.0:1789760783.736408:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762798825726 time 762798565310 state 0x1 ref 1 00000100:00100000:0.0:1789760783.736430:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762798825726 time 762841140228 state 0x0 ref 1 00000100:00100000:0.0:1789760783.736442:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508459392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760783.736593:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24735744, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:0.0:1789760783.736602:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56c30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.736619:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.736624:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.736630:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56c30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.736645:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56c30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760783.771419:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24735744 -> 24739840 00010000:00010000:3.0:1789760783.771437:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56c30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.771450:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56c30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.771459:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56c30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.771466:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760783.771471:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56c30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.771481:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56c30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.771509:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56c30 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.771518:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56c30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24735744->24739839] (req 24735744->24739839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.771602:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508459392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35158us (35463us total) trans 4294973359 rc 0/0 00000100:00100000:3.0:1789760783.771611:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6071 00000100:00100000:3.0:1789760783.771622:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762798825726 time 762841140228 state 0x2 ref 0 00000001:00080000:2.0:1789760783.773339:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973359 is committed 00000100:00100000:0.0:1789760783.776181:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760783.776395:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508459520 00000100:00100000:2.0:1789760783.776404:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508459520 00000100:00100000:2.0:1789760783.776456:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762798825726 time 762841140228 state 0x0 ref 1 00000100:00100000:2.0:1789760783.776471:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762841400644 time 762841140228 state 0x1 ref 1 00000100:00100000:2.0:1789760783.776508:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762841400644 time 762881218778 state 0x0 ref 1 00000100:00100000:2.0:1789760783.776520:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508459520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760783.776573:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24739840, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:2.0:1789760783.776582:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.776604:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760783.776608:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.776614:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.776628:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.814512:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973360 is committed 00002000:00010000:0.0:1789760783.814697:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24739840 -> 24743936 00010000:00010000:0.0:1789760783.814713:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.814733:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.814741:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.814749:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760783.814753:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.814776:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c37 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.814906:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c37 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.814917:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56c37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24739840->24743935] (req 24739840->24743935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.815009:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508459520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38486us (38832us total) trans 4294973360 rc 0/0 00000100:00100000:0.0:1789760783.815019:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6072 00000100:00100000:0.0:1789760783.815029:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762841400644 time 762881218778 state 0x2 ref 0 00000100:00100000:0.0:1789760783.821376:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760783.821584:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508459648 00000100:00100000:2.0:1789760783.821593:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508459648 00000100:00100000:2.0:1789760783.821645:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762841400644 time 762881218778 state 0x0 ref 1 00000100:00100000:2.0:1789760783.821662:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762881479194 time 762881218778 state 0x1 ref 1 00000100:00100000:2.0:1789760783.821691:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762881479194 time 762926401594 state 0x0 ref 1 00000100:00100000:2.0:1789760783.821703:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508459648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760783.821750:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24743936, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:2.0:1789760783.821770:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.821785:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:2.0:1789760783.821789:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.821796:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.821810:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760783.844656:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24743936 -> 24748032 00010000:00010000:2.0:1789760783.844690:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.844707:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.844718:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.844726:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760783.844731:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.844740:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.844773:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.844782:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24743936->24748031] (req 24743936->24748031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.844883:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508459648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23177us (23512us total) trans 4294973361 rc 0/0 00000100:00100000:2.0:1789760783.844894:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6073 00000100:00100000:2.0:1789760783.844905:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762881479194 time 762926401594 state 0x2 ref 0 00000001:00080000:2.0:1789760783.844944:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973361 is committed 00000100:00100000:3.0:1789760783.848654:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760783.848747:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508459776 00000100:00100000:0.0:1789760783.848765:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508459776 00000100:00100000:0.0:1789760783.848819:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762881479194 time 762926401594 state 0x0 ref 1 00000100:00100000:0.0:1789760783.848834:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762926662010 time 762926401594 state 0x1 ref 1 00000100:00100000:0.0:1789760783.848860:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762926662010 time 762953571670 state 0x0 ref 1 00000100:00100000:0.0:1789760783.848872:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888133100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508459776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760783.848917:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24748032, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:0.0:1789760783.848925:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56c45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848941:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.848946:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.848952:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56c45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848966:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56c45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.884399:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973362 is committed 00002000:00010000:1.0:1789760783.884467:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24748032 -> 24752128 00010000:00010000:1.0:1789760783.884483:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56c45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.884494:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56c45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.884503:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56c45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.884510:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760783.884515:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56c45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.884530:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56c45 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.884549:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56c45 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.884557:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56c45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24748032->24752127] (req 24748032->24752127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.884655:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888133100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508459776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35779us (36005us total) trans 4294973362 rc 0/0 00000100:00100000:1.0:1789760783.884665:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6074 00000100:00100000:1.0:1789760783.884674:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762926662010 time 762953571670 state 0x2 ref 0 00000100:00100000:1.0:1789760783.890005:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760783.890090:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508459904 00000100:00100000:0.0:1789760783.890099:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508459904 00000100:00100000:0.0:1789760783.890208:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762926662010 time 762953571670 state 0x0 ref 1 00000100:00100000:0.0:1789760783.890223:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762953832086 time 762953571670 state 0x1 ref 1 00000100:00100000:0.0:1789760783.890256:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762953832086 time 762994966477 state 0x0 ref 1 00000100:00100000:0.0:1789760783.890269:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508459904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760783.890341:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24752128, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:0.0:1789760783.890351:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56c4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.890366:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=762s v=(0 0) 00010000:00010000:0.0:1789760783.890370:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.890376:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56c4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.890389:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56c4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.926978:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973363 is committed 00002000:00010000:2.0:1789760783.927085:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24752128 -> 24756224 00010000:00010000:2.0:1789760783.927222:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56c4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.927236:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56c4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.927246:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56c4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.927253:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760783.927257:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56c4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.927266:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56c4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.927360:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56c4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.927369:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56c4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24752128->24756223] (req 24752128->24756223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.927515:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508459904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37243us (37515us total) trans 4294973363 rc 0/0 00000100:00100000:2.0:1789760783.927528:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6075 00000100:00100000:2.0:1789760783.927538:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762953832086 time 762994966477 state 0x2 ref 0 00000100:00100000:0.0:1789760783.931230:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760783.931431:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508460032 00000100:00100000:2.0:1789760783.931449:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508460032 00000100:00100000:2.0:1789760783.931501:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762953832086 time 762994966477 state 0x0 ref 1 00000100:00100000:2.0:1789760783.931516:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762995226893 time 762994966477 state 0x1 ref 1 00000100:00100000:3.0:1789760783.931599:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762995226893 time 763036306878 state 0x0 ref 1 00000100:00100000:3.0:1789760783.931615:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508460032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760783.931665:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24756224, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:3.0:1789760783.931685:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.931701:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760783.931705:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.931711:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.931724:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760783.969267:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24756224 -> 24760320 00010000:00010000:3.0:1789760783.969304:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.969317:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.969326:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.969334:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760783.969339:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.969348:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.969368:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.969376:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24756224->24760319] (req 24756224->24760319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.969464:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508460032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37849us (38239us total) trans 4294973364 rc 0/0 00000100:00100000:3.0:1789760783.969476:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6076 00000100:00100000:3.0:1789760783.969486:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762995226893 time 763036306878 state 0x2 ref 0 00000001:00080000:1.0:1789760783.969555:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973364 is committed 00000100:00100000:3.0:1789760783.974584:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760783.976043:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508460160 00000100:00100000:1.0:1789760783.976057:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508460160 00000100:00100000:1.0:1789760783.976129:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 762995226893 time 763036306878 state 0x0 ref 1 00000100:00100000:1.0:1789760783.976146:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763036567294 time 763036306878 state 0x1 ref 1 00000100:00100000:1.0:1789760783.976173:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763036567294 time 763080883358 state 0x0 ref 1 00000100:00100000:1.0:1789760783.976187:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508460160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760783.976242:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24760320, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:1.0:1789760783.976252:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56c5a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.976270:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:1.0:1789760783.976275:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.976282:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56c5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.976313:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56c5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760784.009485:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24760320 -> 24764416 00010000:00010000:2.0:1789760784.009504:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56c5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.009518:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56c5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.009527:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56c5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.009534:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760784.009539:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56c5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.009549:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56c5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.009568:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56c5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.009575:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56c5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24760320->24764415] (req 24760320->24764415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.009670:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508460160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33481us (35093us total) trans 4294973365 rc 0/0 00000100:00100000:2.0:1789760784.009680:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6077 00000100:00100000:2.0:1789760784.009690:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763036567294 time 763080883358 state 0x2 ref 0 00000001:00080000:3.0:1789760784.010281:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973365 is committed 00000100:00100000:1.0:1789760784.015822:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760784.016638:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508460288 00000100:00100000:0.0:1789760784.016648:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508460288 00000100:00100000:0.0:1789760784.016703:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763036567294 time 763080883358 state 0x0 ref 1 00000100:00100000:0.0:1789760784.016719:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763081143774 time 763080883358 state 0x1 ref 1 00000100:00100000:0.0:1789760784.016763:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763081143774 time 763121464615 state 0x0 ref 1 00000100:00100000:0.0:1789760784.016776:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508460288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760784.016823:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24764416, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:0.0:1789760784.016832:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56c61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.016849:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760784.016853:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.016858:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56c61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.016873:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56c61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.049810:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24764416 -> 24768512 00010000:00010000:0.0:1789760784.049832:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56c61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.049846:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56c61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.049855:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56c61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.049863:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760784.049868:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56c61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.049876:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56c61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.049896:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56c61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.049903:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56c61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24764416->24768511] (req 24764416->24768511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.050004:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508460288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33225us (34187us total) trans 4294973366 rc 0/0 00000100:00100000:0.0:1789760784.050015:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6078 00000100:00100000:0.0:1789760784.050025:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763081143774 time 763121464615 state 0x2 ref 0 00000001:00080000:2.0:1789760784.050922:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973366 is committed 00000100:00100000:0.0:1789760784.057016:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760784.057113:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508460416 00000100:00100000:3.0:1789760784.057124:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508460416 00000100:00100000:3.0:1789760784.057178:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763081143774 time 763121464615 state 0x0 ref 1 00000100:00100000:3.0:1789760784.057193:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763121725031 time 763121464615 state 0x1 ref 1 00000100:00100000:3.0:1789760784.057240:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763121725031 time 763161949632 state 0x0 ref 1 00000100:00100000:3.0:1789760784.057252:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508460416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760784.057476:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24768512, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:3.0:1789760784.057487:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.057502:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760784.057507:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.057513:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.057527:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.114945:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973367 is committed 00002000:00010000:0.0:1789760784.115114:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24768512 -> 24772608 00010000:00010000:0.0:1789760784.115140:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.115153:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.115163:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.115192:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760784.115197:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.115214:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.115238:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.115245:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe56c68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24768512->24772607] (req 24768512->24772607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.115374:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508460416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58118us (58362us total) trans 4294973367 rc 0/0 00000100:00100000:0.0:1789760784.115386:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6079 00000100:00100000:0.0:1789760784.115396:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763121725031 time 763161949632 state 0x2 ref 0 00000100:00100000:3.0:1789760784.124587:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760784.124695:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508460544 00000100:00100000:2.0:1789760784.124704:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508460544 00000100:00100000:2.0:1789760784.124769:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763121725031 time 763161949632 state 0x0 ref 1 00000100:00100000:2.0:1789760784.124785:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763162210048 time 763161949632 state 0x1 ref 1 00000100:00100000:2.0:1789760784.124824:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763162210048 time 763229534840 state 0x0 ref 1 00000100:00100000:2.0:1789760784.124837:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508460544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760784.124887:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24772608, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:2.0:1789760784.124897:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.124913:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.124918:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.124924:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.124939:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.126435:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760784.166562:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973368 is committed 00002000:00010000:2.0:1789760784.167920:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24772608 -> 24776704 00010000:00010000:2.0:1789760784.167939:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.167953:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.167963:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.167970:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760784.167975:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.167984:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.168009:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.168016:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24772608->24776703] (req 24772608->24776703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.168165:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508460544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43325us (43582us total) trans 4294973368 rc 0/0 00000100:00100000:2.0:1789760784.168176:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6080 00000100:00100000:2.0:1789760784.168186:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763162210048 time 763229534840 state 0x2 ref 0 00000100:00100000:1.0:1789760784.179570:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760784.179684:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508460672 00000100:00100000:0.0:1789760784.179694:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508460672 00000100:00100000:0.0:1789760784.179747:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763162210048 time 763229534840 state 0x0 ref 1 00000100:00100000:0.0:1789760784.179775:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763229795256 time 763229534840 state 0x1 ref 1 00000100:00100000:0.0:1789760784.179804:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763229795256 time 763284514631 state 0x0 ref 1 00000100:00100000:0.0:1789760784.179816:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508460672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760784.179864:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24776704, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:0.0:1789760784.179873:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56c76 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.179895:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:0.0:1789760784.179900:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.179906:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56c76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.179920:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56c76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760784.231140:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24776704 -> 24780800 00010000:00010000:2.0:1789760784.231157:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56c76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.231179:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56c76 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.231188:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56c76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.231196:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760784.231211:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56c76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.231225:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56c76 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.231244:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56c76 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.231252:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56c76 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24776704->24780799] (req 24776704->24780799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.231384:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508460672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51563us (51818us total) trans 4294973369 rc 0/0 00000100:00100000:2.0:1789760784.231394:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6081 00000100:00100000:2.0:1789760784.231403:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763229795256 time 763284514631 state 0x2 ref 0 00000001:00080000:3.0:1789760784.237442:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973369 is committed 00000100:00100000:0.0:1789760784.239010:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760784.239407:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508460800 00000100:00100000:3.0:1789760784.239417:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508460800 00000100:00100000:3.0:1789760784.239468:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763229795256 time 763284514631 state 0x0 ref 1 00000100:00100000:3.0:1789760784.239483:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763284775047 time 763284514631 state 0x1 ref 1 00000100:00100000:3.0:1789760784.239561:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763284775047 time 763344269836 state 0x0 ref 1 00000100:00100000:3.0:1789760784.239576:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508460800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760784.239625:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24780800, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:3.0:1789760784.239634:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.239650:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760784.239654:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.239661:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.239686:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.276814:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973370 is committed 00002000:00010000:3.0:1789760784.276919:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24780800 -> 24784896 00010000:00010000:3.0:1789760784.276935:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.276948:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.276957:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.276965:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760784.276970:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.276980:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c7d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.277000:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c7d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.277008:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56c7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24780800->24784895] (req 24780800->24784895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.277115:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508460800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37530us (38102us total) trans 4294973370 rc 0/0 00000100:00100000:3.0:1789760784.277127:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6082 00000100:00100000:3.0:1789760784.277136:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763284775047 time 763344269836 state 0x2 ref 0 00000100:00100000:3.0:1789760784.280684:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760784.280819:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508460928 00000100:00100000:2.0:1789760784.280828:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508460928 00000100:00100000:2.0:1789760784.280881:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763284775047 time 763344269836 state 0x0 ref 1 00000100:00100000:2.0:1789760784.280896:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763344530252 time 763344269836 state 0x1 ref 1 00000100:00100000:2.0:1789760784.280928:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763344530252 time 763385638587 state 0x0 ref 1 00000100:00100000:2.0:1789760784.280940:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508460928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760784.280986:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24784896, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:2.0:1789760784.280995:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.281010:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.281014:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.281020:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.281034:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760784.316231:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24784896 -> 24788992 00010000:00010000:2.0:1789760784.316252:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.316266:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.316276:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.316301:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760784.316307:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.316317:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.316336:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.316345:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24784896->24788991] (req 24784896->24788991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.316475:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973371 is committed 00000100:00100000:2.0:1789760784.316625:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508460928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35682us (35954us total) trans 4294973371 rc 0/0 00000100:00100000:2.0:1789760784.316635:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6083 00000100:00100000:2.0:1789760784.316645:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763344530252 time 763385638587 state 0x2 ref 0 00000100:00100000:0.0:1789760784.329277:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760784.329449:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508461056 00000100:00100000:2.0:1789760784.329459:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508461056 00000100:00100000:2.0:1789760784.329512:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763344530252 time 763385638587 state 0x0 ref 1 00000100:00100000:2.0:1789760784.329528:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763385899003 time 763385638587 state 0x1 ref 1 00000100:00100000:2.0:1789760784.329558:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763385899003 time 763434269371 state 0x0 ref 1 00000100:00100000:2.0:1789760784.329571:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508461056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760784.329618:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24788992, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:2.0:1789760784.329627:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.329643:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.329647:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.329653:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.329806:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760784.375580:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24788992 -> 24793088 00010000:00010000:2.0:1789760784.375611:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.375627:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.375636:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.375643:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973372 is committed 00010000:00010000:2.0:1789760784.375644:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760784.375650:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.375659:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.375681:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.375690:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24788992->24793087] (req 24788992->24793087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.375804:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508461056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46231us (46533us total) trans 4294973372 rc 0/0 00000100:00100000:2.0:1789760784.375815:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6084 00000100:00100000:2.0:1789760784.375824:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763385899003 time 763434269371 state 0x2 ref 0 00000100:00100000:2.0:1789760784.384891:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760784.385001:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508461184 00000100:00100000:3.0:1789760784.385011:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508461184 00000100:00100000:3.0:1789760784.385066:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763385899003 time 763434269371 state 0x0 ref 1 00000100:00100000:3.0:1789760784.385080:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763434529787 time 763434269371 state 0x1 ref 1 00000100:00100000:3.0:1789760784.385115:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763434529787 time 763489825506 state 0x0 ref 1 00000100:00100000:3.0:1789760784.385128:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508461184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760784.385184:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24793088, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:3.0:1789760784.385194:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56c92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.385208:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760784.385213:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.385219:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56c92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.385233:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56c92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.412697:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24793088 -> 24797184 00010000:00010000:0.0:1789760784.412717:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56c92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.412730:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56c92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.412739:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56c92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.412747:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760784.412763:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56c92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.412773:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56c92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.412796:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56c92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.412804:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe56c92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24793088->24797183] (req 24793088->24797183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.412894:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508461184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27764us (28007us total) trans 4294973373 rc 0/0 00000100:00100000:0.0:1789760784.412905:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6085 00000100:00100000:0.0:1789760784.412915:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763434529787 time 763489825506 state 0x2 ref 0 00000001:00080000:2.0:1789760784.414867:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973373 is committed 00000100:00100000:3.0:1789760784.416727:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760784.416839:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508461312 00000100:00100000:2.0:1789760784.416848:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508461312 00000100:00100000:2.0:1789760784.416903:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763434529787 time 763489825506 state 0x0 ref 1 00000100:00100000:2.0:1789760784.416919:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763490085922 time 763489825506 state 0x1 ref 1 00000100:00100000:2.0:1789760784.416951:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763490085922 time 763521661361 state 0x0 ref 1 00000100:00100000:2.0:1789760784.416962:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508461312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760784.417024:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24797184, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:2.0:1789760784.417034:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.417051:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.417055:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.417061:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.417082:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.451431:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973374 is committed 00002000:00010000:2.0:1789760784.451652:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24797184 -> 24801280 00010000:00010000:2.0:1789760784.451738:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.451761:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.451771:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.451778:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760784.451783:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.451792:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.451870:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.451881:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56c99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24797184->24801279] (req 24797184->24801279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.451983:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508461312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35016us (35259us total) trans 4294973374 rc 0/0 00000100:00100000:2.0:1789760784.451993:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6086 00000100:00100000:2.0:1789760784.452002:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763490085922 time 763521661361 state 0x2 ref 0 00000100:00100000:3.0:1789760784.455475:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760784.455805:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508461440 00000100:00100000:2.0:1789760784.455816:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508461440 00000100:00100000:2.0:1789760784.455869:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763490085922 time 763521661361 state 0x0 ref 1 00000100:00100000:2.0:1789760784.455884:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763521921777 time 763521661361 state 0x1 ref 1 00000100:00100000:2.0:1789760784.455916:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763521921777 time 763560626453 state 0x0 ref 1 00000100:00100000:2.0:1789760784.455928:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508461440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760784.455978:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24801280, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:2.0:1789760784.455987:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56ca0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.456002:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.456007:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.456013:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56ca0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.456027:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56ca0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760784.488702:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24801280 -> 24805376 00010000:00010000:0.0:1789760784.488721:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56ca0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.488734:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56ca0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.488742:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56ca0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.488748:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760784.488761:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56ca0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.488776:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56ca0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.488794:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56ca0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.488801:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56ca0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24801280->24805375] (req 24801280->24805375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.488894:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508461440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32962us (33423us total) trans 4294973375 rc 0/0 00000100:00100000:0.0:1789760784.488905:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6087 00000100:00100000:0.0:1789760784.488916:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763521921777 time 763560626453 state 0x2 ref 0 00000001:00080000:1.0:1789760784.492926:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973375 is committed 00000100:00100000:0.0:1789760784.493931:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760784.494022:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508461568 00000100:00100000:3.0:1789760784.494031:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508461568 00000100:00100000:3.0:1789760784.494088:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763521921777 time 763560626453 state 0x0 ref 1 00000100:00100000:3.0:1789760784.494110:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763560886869 time 763560626453 state 0x1 ref 1 00000100:00100000:3.0:1789760784.494140:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763560886869 time 763598850179 state 0x0 ref 1 00000100:00100000:3.0:1789760784.494152:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508461568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760784.494203:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24805376, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0xc007 00010000:00010000:3.0:1789760784.494213:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ca7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.494231:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760784.494235:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.494242:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ca7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.494256:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ca7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.570221:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973376 is committed 00002000:00010000:3.0:1789760784.570331:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24805376 -> 24809472 00002000:00010000:3.0:1789760784.570336:0:14638:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760784 -> 1789760785 00002000:00010000:3.0:1789760784.570338:0:14638:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760784 -> 1789760785 00010000:00010000:3.0:1789760784.570352:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ca7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.570363:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ca7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.570371:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ca7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.570378:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760784.570382:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ca7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.570391:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ca7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.570411:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ca7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.570419:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ca7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24805376->24809471] (req 24805376->24809471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.570510:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508461568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 76355us (76580us total) trans 4294973376 rc 0/0 00000100:00100000:3.0:1789760784.570519:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6088 00000100:00100000:3.0:1789760784.570529:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763560886869 time 763598850179 state 0x2 ref 0 00000100:00100000:3.0:1789760784.574064:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760784.574235:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508461696 00000100:00100000:2.0:1789760784.574244:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508461696 00000100:00100000:2.0:1789760784.574335:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763560886869 time 763598850179 state 0x0 ref 1 00000100:00100000:2.0:1789760784.574350:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763599110595 time 763598850179 state 0x1 ref 1 00000100:00100000:2.0:1789760784.574460:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763599110595 time 763679170585 state 0x0 ref 1 00000100:00100000:2.0:1789760784.574473:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508461696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760784.574520:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24809472, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:2.0:1789760784.574528:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.574542:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.574547:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.574553:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.574566:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.631948:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973377 is committed 00002000:00010000:3.0:1789760784.636029:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24809472 -> 24813568 00010000:00010000:3.0:1789760784.636051:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.636063:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.636072:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.636078:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760784.636083:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.636092:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.636112:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.636120:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24809472->24813567] (req 24809472->24813567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.636225:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508461696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61749us (62166us total) trans 4294973377 rc 0/0 00000100:00100000:3.0:1789760784.636236:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6089 00000100:00100000:3.0:1789760784.636245:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763599110595 time 763679170585 state 0x2 ref 0 00000100:00100000:0.0:1789760784.647129:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760784.647212:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508461824 00000100:00100000:2.0:1789760784.647222:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508461824 00000100:00100000:2.0:1789760784.647279:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763599110595 time 763679170585 state 0x0 ref 1 00000100:00100000:2.0:1789760784.647345:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763679431001 time 763679170585 state 0x1 ref 1 00000100:00100000:2.0:1789760784.647378:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763679431001 time 763752089111 state 0x0 ref 1 00000100:00100000:2.0:1789760784.647392:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508461824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760784.647439:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24813568, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:2.0:1789760784.647448:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cb5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.647462:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.647467:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.647473:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.647487:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760784.725945:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24813568 -> 24817664 00010000:00010000:2.0:1789760784.725966:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.725979:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cb5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.725989:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.725997:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760784.726003:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.726013:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cb5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.726036:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cb5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.726044:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cb5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24813568->24817663] (req 24813568->24817663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.726150:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508461824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 78755us (79024us total) trans 4294973378 rc 0/0 00000100:00100000:2.0:1789760784.726160:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6090 00000100:00100000:2.0:1789760784.726171:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763679431001 time 763752089111 state 0x2 ref 0 00000001:00080000:0.0:1789760784.728321:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973378 is committed 00000100:00100000:0.0:1789760784.732953:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760784.733775:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508461952 00000100:00100000:3.0:1789760784.733787:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508461952 00000100:00100000:3.0:1789760784.733845:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763679431001 time 763752089111 state 0x0 ref 1 00000100:00100000:3.0:1789760784.733860:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763752349527 time 763752089111 state 0x1 ref 1 00000100:00100000:3.0:1789760784.733896:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763752349527 time 763838605928 state 0x0 ref 1 00000100:00100000:3.0:1789760784.733908:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508461952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760784.733957:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24817664, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:3.0:1789760784.733967:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56cbc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.733984:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:3.0:1789760784.733989:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.733995:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56cbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.734009:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56cbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.778086:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973379 is committed 00002000:00010000:2.0:1789760784.778329:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24817664 -> 24821760 00010000:00010000:2.0:1789760784.778344:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56cbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.778356:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56cbc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.778364:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56cbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.778377:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760784.778382:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56cbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.778391:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56cbc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.778411:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56cbc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.778419:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe56cbc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24817664->24821759] (req 24817664->24821759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.778593:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508461952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44681us (45644us total) trans 4294973379 rc 0/0 00000100:00100000:2.0:1789760784.778604:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6091 00000100:00100000:2.0:1789760784.778615:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763752349527 time 763838605928 state 0x2 ref 0 00000100:00100000:3.0:1789760784.782466:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760784.782578:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508462080 00000100:00100000:2.0:1789760784.782588:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508462080 00000100:00100000:2.0:1789760784.782642:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763752349527 time 763838605928 state 0x0 ref 1 00000100:00100000:2.0:1789760784.782657:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763838866344 time 763838605928 state 0x1 ref 1 00000100:00100000:2.0:1789760784.782694:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763838866344 time 763887404446 state 0x0 ref 1 00000100:00100000:2.0:1789760784.782710:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508462080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760784.782769:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24821760, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:2.0:1789760784.782786:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cc3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.782804:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.782808:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.782815:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.782829:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.844603:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973380 is committed 00002000:00010000:2.0:1789760784.844685:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24821760 -> 24825856 00010000:00010000:2.0:1789760784.844700:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.844713:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.844722:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.844730:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760784.844734:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.844744:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cc3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.844773:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cc3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.844781:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56cc3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24821760->24825855] (req 24821760->24825855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.844866:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508462080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 62154us (62405us total) trans 4294973380 rc 0/0 00000100:00100000:2.0:1789760784.844877:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6092 00000100:00100000:2.0:1789760784.844886:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763838866344 time 763887404446 state 0x2 ref 0 00000100:00100000:0.0:1789760784.849366:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760784.849439:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508462208 00000100:00100000:2.0:1789760784.849448:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508462208 00000100:00100000:2.0:1789760784.849509:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763838866344 time 763887404446 state 0x0 ref 1 00000100:00100000:2.0:1789760784.849524:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763887664862 time 763887404446 state 0x1 ref 1 00000100:00100000:2.0:1789760784.849556:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763887664862 time 763954266232 state 0x0 ref 1 00000100:00100000:2.0:1789760784.849571:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508462208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760784.849624:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24825856, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:2.0:1789760784.849636:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.849655:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=763s v=(0 0) 00010000:00010000:2.0:1789760784.849669:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.849678:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.849695:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.885837:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973381 is committed 00002000:00010000:2.0:1789760784.885942:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24825856 -> 24829952 00010000:00010000:2.0:1789760784.885959:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.885973:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.885982:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.885989:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760784.885994:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.886003:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.886022:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.886031:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24825856->24829951] (req 24825856->24829951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.886136:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508462208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36565us (36776us total) trans 4294973381 rc 0/0 00000100:00100000:2.0:1789760784.886145:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6093 00000100:00100000:2.0:1789760784.886156:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763887664862 time 763954266232 state 0x2 ref 0 00000100:00100000:2.0:1789760784.901189:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760784.902027:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508462336 00000100:00100000:3.0:1789760784.902038:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508462336 00000100:00100000:3.0:1789760784.902090:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763887664862 time 763954266232 state 0x0 ref 1 00000100:00100000:3.0:1789760784.902104:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763954526648 time 763954266232 state 0x1 ref 1 00000100:00100000:3.0:1789760784.902135:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763954526648 time 764006846082 state 0x0 ref 1 00000100:00100000:3.0:1789760784.902146:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508462336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760784.902190:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24829952, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:3.0:1789760784.902199:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56cd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.902215:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760784.902219:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.902226:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56cd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.902242:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56cd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.987922:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973382 is committed 00002000:00010000:3.0:1789760784.988469:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24829952 -> 24834048 00010000:00010000:3.0:1789760784.988484:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56cd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.988497:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56cd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.988506:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56cd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.988513:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760784.988518:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56cd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.988526:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56cd1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.988544:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56cd1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.988552:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56cd1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24829952->24834047] (req 24829952->24834047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.988643:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508462336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 86494us (87469us total) trans 4294973382 rc 0/0 00000100:00100000:3.0:1789760784.988654:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6094 00000100:00100000:3.0:1789760784.988665:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763954526648 time 764006846082 state 0x2 ref 0 00000100:00100000:3.0:1789760784.991890:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760784.992020:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508462464 00000100:00100000:2.0:1789760784.992142:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508462464 00000100:00100000:2.0:1789760784.992200:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 763954526648 time 764006846082 state 0x0 ref 1 00000100:00100000:2.0:1789760784.992216:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764007106498 time 764006846082 state 0x1 ref 1 00000100:00100000:2.0:1789760784.992428:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764007106498 time 764097137114 state 0x0 ref 1 00000100:00100000:2.0:1789760784.992442:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508462464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760784.992489:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24834048, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:2.0:1789760784.992498:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cd8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.992515:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760784.992519:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.992526:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.992539:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.031197:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973383 is committed 00002000:00010000:3.0:1789760785.031281:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24834048 -> 24838144 00010000:00010000:3.0:1789760785.031322:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.031336:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.031346:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.031354:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760785.031359:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.031369:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.031388:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cd8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.031402:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56cd8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24834048->24838143] (req 24834048->24838143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.031536:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508462464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39091us (39649us total) trans 4294973383 rc 0/0 00000100:00100000:3.0:1789760785.031546:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6095 00000100:00100000:3.0:1789760785.031555:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764007106498 time 764097137114 state 0x2 ref 0 00000100:00100000:0.0:1789760785.034910:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760785.035040:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508462592 00000100:00100000:2.0:1789760785.035050:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508462592 00000100:00080000:3.0:1789760785.035197:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760785.035234:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764007106498 time 764097137114 state 0x0 ref 1 00000100:00100000:2.0:1789760785.035250:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764097397530 time 764097137114 state 0x1 ref 1 00000100:00100000:2.0:1789760785.035281:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764097397530 time 764139991045 state 0x0 ref 1 00000100:00100000:2.0:1789760785.035363:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508462592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760785.035410:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24838144, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:2.0:1789760785.035420:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56cdf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.035435:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760785.035439:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.035446:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56cdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.035461:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56cdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.083356:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973384 is committed 00002000:00010000:2.0:1789760785.083454:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24838144 -> 24842240 00010000:00010000:2.0:1789760785.083469:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56cdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.083482:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56cdf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.083490:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56cdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.083496:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760785.083501:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56cdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.083509:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56cdf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.083529:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56cdf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.083537:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56cdf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24838144->24842239] (req 24838144->24842239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.083633:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508462592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48269us (48727us total) trans 4294973384 rc 0/0 00000100:00100000:2.0:1789760785.083646:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6096 00000100:00100000:2.0:1789760785.083657:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764097397530 time 764139991045 state 0x2 ref 0 00000100:00100000:2.0:1789760785.094436:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760785.094534:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508462720 00000100:00100000:3.0:1789760785.094544:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508462720 00000100:00100000:3.0:1789760785.094598:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764097397530 time 764139991045 state 0x0 ref 1 00000100:00100000:3.0:1789760785.094614:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764140251461 time 764139991045 state 0x1 ref 1 00000100:00100000:3.0:1789760785.094690:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764140251461 time 764199399720 state 0x0 ref 1 00000100:00100000:3.0:1789760785.094702:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508462720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760785.094766:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24842240, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:3.0:1789760785.094775:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ce6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.094789:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.094793:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.094799:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ce6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.094812:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ce6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.133771:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973385 is committed 00002000:00010000:0.0:1789760785.134093:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24842240 -> 24846336 00010000:00010000:0.0:1789760785.134125:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ce6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.134143:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ce6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.134152:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ce6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.134158:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760785.134164:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ce6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.134173:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ce6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.134193:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ce6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.134200:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe56ce6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24842240->24846335] (req 24842240->24846335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.134566:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508462720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39859us (40133us total) trans 4294973385 rc 0/0 00000100:00100000:0.0:1789760785.134576:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6097 00000100:00100000:0.0:1789760785.134585:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764140251461 time 764199399720 state 0x2 ref 0 00000100:00100000:3.0:1789760785.142617:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760785.145418:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508462848 00000100:00100000:2.0:1789760785.145432:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508462848 00000100:00100000:3.0:1789760785.145596:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764140251461 time 764199399720 state 0x0 ref 1 00000100:00100000:3.0:1789760785.145612:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764199660136 time 764199399720 state 0x1 ref 1 00000100:00100000:3.0:1789760785.145693:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764199660136 time 764250402351 state 0x0 ref 1 00000100:00100000:3.0:1789760785.145706:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508462848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760785.145765:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24846336, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:3.0:1789760785.145774:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56ced lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.145789:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.145793:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.145799:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56ced lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.145813:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56ced lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.164693:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760785.210901:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973386 is committed 00002000:00010000:3.0:1789760785.211622:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24846336 -> 24850432 00010000:00010000:3.0:1789760785.211640:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56ced lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.211653:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56ced lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.211662:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56ced lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.211670:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760785.211674:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56ced lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.211683:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56ced lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.211701:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56ced lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.211710:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56ced lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24846336->24850431] (req 24846336->24850431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.211817:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508462848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 66109us (69206us total) trans 4294973386 rc 0/0 00000100:00100000:3.0:1789760785.211826:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6098 00000100:00100000:3.0:1789760785.211835:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764199660136 time 764250402351 state 0x2 ref 0 00000100:00100000:3.0:1789760785.225270:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760785.225450:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508462976 00000100:00100000:2.0:1789760785.225459:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508462976 00000100:00100000:2.0:1789760785.225517:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764199660136 time 764250402351 state 0x0 ref 1 00000100:00100000:2.0:1789760785.225532:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764250662767 time 764250402351 state 0x1 ref 1 00000100:00100000:2.0:1789760785.225567:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764250662767 time 764330276399 state 0x0 ref 1 00000100:00100000:2.0:1789760785.225580:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888132a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508462976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760785.225628:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24850432, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:2.0:1789760785.225638:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56cf4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.225654:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760785.225659:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.225665:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56cf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.225679:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56cf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.273782:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973387 is committed 00002000:00010000:3.0:1789760785.273895:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24850432 -> 24854528 00010000:00010000:3.0:1789760785.273911:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56cf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.273923:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56cf4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.273931:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56cf4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.273938:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760785.273943:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56cf4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.273952:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56cf4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.273971:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56cf4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.273979:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56cf4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24850432->24854527] (req 24850432->24854527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.274364:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888132a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508462976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48781us (49097us total) trans 4294973387 rc 0/0 00000100:00100000:3.0:1789760785.274375:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6099 00000100:00100000:3.0:1789760785.274384:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764250662767 time 764330276399 state 0x2 ref 0 00000100:00100000:2.0:1789760785.277956:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760785.278610:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b66c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534088832:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760785.278683:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760785.278766:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534088832 00000100:00100000:1.0:1789760785.278774:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534088832 00000100:00100000:1.0:1789760785.278815:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:1.0:1789760785.278824:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534088832:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760785.278861:0:7358:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760785.278905:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534088832:12345-0@lo:41:osp-pre-1-0.0 Request processed in 82us (222us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760785.278913:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:1.0:1789760785.278958:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b66c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534088832:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760785.279538:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508463104 00000100:00100000:0.0:1789760785.279545:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508463104 00000100:00100000:1.0:1789760785.279772:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764250662767 time 764330276399 state 0x0 ref 1 00000100:00100000:1.0:1789760785.279785:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764330536815 time 764330276399 state 0x1 ref 1 00000100:00100000:1.0:1789760785.279848:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764330536815 time 764384558676 state 0x0 ref 1 00000100:00100000:1.0:1789760785.279860:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508463104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760785.279904:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24854528, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:1.0:1789760785.279913:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56cfb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.279928:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760785.279932:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.279939:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56cfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.279952:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56cfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.334807:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973388 is committed 00002000:00010000:3.0:1789760785.334980:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24854528 -> 24858624 00010000:00010000:3.0:1789760785.334995:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56cfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.335008:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56cfb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.335016:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56cfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.335023:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760785.335027:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56cfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.335036:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56cfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.335056:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56cfb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.335063:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56cfb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24854528->24858623] (req 24854528->24858623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.335170:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508463104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55307us (57218us total) trans 4294973388 rc 0/0 00000100:00100000:3.0:1789760785.335180:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6100 00000100:00100000:3.0:1789760785.335190:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764330536815 time 764384558676 state 0x2 ref 0 00000100:00100000:0.0:1789760785.339772:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760785.339856:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508463232 00000100:00100000:3.0:1789760785.339866:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508463232 00000100:00100000:3.0:1789760785.339922:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764330536815 time 764384558676 state 0x0 ref 1 00000100:00100000:3.0:1789760785.339937:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764384819092 time 764384558676 state 0x1 ref 1 00000100:00100000:3.0:1789760785.339967:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764384819092 time 764444677931 state 0x0 ref 1 00000100:00100000:3.0:1789760785.339979:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508463232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760785.340029:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24858624, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:3.0:1789760785.340039:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56d02 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.340055:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.340059:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.340066:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56d02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.340081:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56d02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.390849:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973389 is committed 00002000:00010000:0.0:1789760785.390946:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24858624 -> 24862720 00010000:00010000:0.0:1789760785.390962:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56d02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.390975:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56d02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.390984:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56d02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.390992:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760785.390997:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56d02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.391006:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56d02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.391026:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56d02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.391034:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe56d02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24858624->24862719] (req 24858624->24862719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.391136:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508463232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51153us (51367us total) trans 4294973389 rc 0/0 00000100:00100000:0.0:1789760785.391146:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6101 00000100:00100000:0.0:1789760785.391156:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764384819092 time 764444677931 state 0x2 ref 0 00000100:00100000:0.0:1789760785.396559:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760785.396677:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508463360 00000100:00100000:3.0:1789760785.396688:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508463360 00000100:00100000:3.0:1789760785.396756:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764384819092 time 764444677931 state 0x0 ref 1 00000100:00100000:3.0:1789760785.396773:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764444938347 time 764444677931 state 0x1 ref 1 00000100:00100000:3.0:1789760785.396803:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764444938347 time 764501513418 state 0x0 ref 1 00000100:00100000:3.0:1789760785.396816:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508463360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760785.396867:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24862720, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:3.0:1789760785.396876:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56d09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.396893:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.396898:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.396904:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56d09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.396920:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56d09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.452996:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973390 is committed 00002000:00010000:0.0:1789760785.453192:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24862720 -> 24866816 00010000:00010000:0.0:1789760785.453209:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56d09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.453221:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56d09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.453497:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56d09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.453505:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760785.453510:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56d09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.453519:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56d09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.453537:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56d09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.453545:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe56d09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24862720->24866815] (req 24862720->24866815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.453751:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508463360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56925us (57190us total) trans 4294973390 rc 0/0 00000100:00100000:0.0:1789760785.453761:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6102 00000100:00100000:0.0:1789760785.453772:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764444938347 time 764501513418 state 0x2 ref 0 00000100:00100000:2.0:1789760785.459199:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760785.459467:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508463488 00000100:00100000:3.0:1789760785.459476:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508463488 00000100:00100000:3.0:1789760785.459530:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764444938347 time 764501513418 state 0x0 ref 1 00000100:00100000:3.0:1789760785.459545:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764501773834 time 764501513418 state 0x1 ref 1 00000100:00100000:3.0:1789760785.459615:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764501773834 time 764564324656 state 0x0 ref 1 00000100:00100000:3.0:1789760785.459628:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508463488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760785.459684:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24866816, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:3.0:1789760785.459694:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56d10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.459709:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.459713:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.459720:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56d10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.459735:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56d10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.525264:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973391 is committed 00002000:00010000:0.0:1789760785.525370:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24866816 -> 24870912 00010000:00010000:0.0:1789760785.525385:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56d10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.525397:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56d10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.525405:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56d10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.525411:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760785.525416:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56d10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.525425:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56d10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.525445:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56d10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.525459:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe56d10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24866816->24870911] (req 24866816->24870911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.525549:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508463488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 65918us (66357us total) trans 4294973391 rc 0/0 00000100:00100000:0.0:1789760785.525559:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6103 00000100:00100000:0.0:1789760785.525570:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764501773834 time 764564324656 state 0x2 ref 0 00000100:00100000:1.0:1789760785.529614:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760785.529757:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508463616 00000100:00100000:0.0:1789760785.529768:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508463616 00000100:00100000:0.0:1789760785.529822:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764501773834 time 764564324656 state 0x0 ref 1 00000100:00100000:0.0:1789760785.529838:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764564585072 time 764564324656 state 0x1 ref 1 00000100:00100000:0.0:1789760785.529907:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764564585072 time 764634615937 state 0x0 ref 1 00000100:00100000:0.0:1789760785.529920:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508463616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760785.529970:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24870912, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0xc007 00010000:00010000:0.0:1789760785.529978:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56d17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.529993:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760785.529997:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.530003:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56d17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.530022:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56d17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.559655:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973392 is committed 00002000:00010000:1.0:1789760785.559738:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24870912 -> 24875008 00002000:00010000:1.0:1789760785.559751:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760785 -> 1789760786 00002000:00010000:1.0:1789760785.559754:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760785 -> 1789760786 00010000:00010000:1.0:1789760785.559767:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56d17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.559779:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56d17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.559789:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56d17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.559796:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760785.559800:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56d17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.559809:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56d17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.559829:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56d17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.559837:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56d17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24870912->24875007] (req 24870912->24875007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.559963:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508463616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30040us (30354us total) trans 4294973392 rc 0/0 00000100:00100000:1.0:1789760785.559973:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6104 00000100:00100000:1.0:1789760785.559983:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764564585072 time 764634615937 state 0x2 ref 0 00000100:00100000:3.0:1789760785.563029:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760785.563134:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508463744 00000100:00100000:0.0:1789760785.563144:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508463744 00000100:00100000:0.0:1789760785.563199:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764564585072 time 764634615937 state 0x0 ref 1 00000100:00100000:0.0:1789760785.563215:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764634876353 time 764634615937 state 0x1 ref 1 00000100:00100000:0.0:1789760785.563245:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764634876353 time 764667955557 state 0x0 ref 1 00000100:00100000:0.0:1789760785.563256:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508463744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760785.563332:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24875008, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:0.0:1789760785.563340:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.563356:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760785.563359:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.563365:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.563380:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760785.587871:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24875008 -> 24879104 00010000:00010000:0.0:1789760785.587889:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.587902:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.587911:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.587918:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760785.587923:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.587932:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.587950:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.587957:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24875008->24879103] (req 24875008->24879103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.588039:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508463744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24781us (25016us total) trans 4294973393 rc 0/0 00000100:00100000:0.0:1789760785.588050:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6105 00000100:00100000:0.0:1789760785.588060:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764634876353 time 764667955557 state 0x2 ref 0 00000001:00080000:2.0:1789760785.588230:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973393 is committed 00000100:00100000:2.0:1789760785.595035:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760785.595810:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508463872 00000100:00100000:0.0:1789760785.595821:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508463872 00000100:00100000:0.0:1789760785.595874:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764634876353 time 764667955557 state 0x0 ref 1 00000100:00100000:0.0:1789760785.595890:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764668215973 time 764667955557 state 0x1 ref 1 00000100:00100000:0.0:1789760785.595910:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764668215973 time 764700621057 state 0x0 ref 1 00000100:00100000:0.0:1789760785.595923:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508463872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760785.595967:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24879104, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:0.0:1789760785.595976:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.595992:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760785.595996:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.596003:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.596018:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760785.639572:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24879104 -> 24883200 00010000:00010000:3.0:1789760785.639596:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.639609:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.639617:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.639624:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760785.639629:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.639638:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.639658:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.639666:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56d25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24879104->24883199] (req 24879104->24883199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.639800:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508463872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43874us (44770us total) trans 4294973394 rc 0/0 00000100:00100000:3.0:1789760785.639810:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6106 00000100:00100000:3.0:1789760785.639821:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764668215973 time 764700621057 state 0x2 ref 0 00000001:00080000:1.0:1789760785.639894:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973394 is committed 00000100:00100000:0.0:1789760785.643476:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760785.643547:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508464000 00000100:00100000:1.0:1789760785.643556:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508464000 00000100:00100000:1.0:1789760785.643606:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764668215973 time 764700621057 state 0x0 ref 1 00000100:00100000:1.0:1789760785.643620:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764700881473 time 764700621057 state 0x1 ref 1 00000100:00100000:1.0:1789760785.643650:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764700881473 time 764748361117 state 0x0 ref 1 00000100:00100000:1.0:1789760785.643669:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508464000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760785.643713:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24883200, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:1.0:1789760785.643721:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.643737:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760785.643751:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.643758:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.643772:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.682648:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973395 is committed 00002000:00010000:1.0:1789760785.682823:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24883200 -> 24887296 00010000:00010000:1.0:1789760785.682838:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.682857:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.682866:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.682874:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760785.682879:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.682888:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.682908:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.682916:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24883200->24887295] (req 24883200->24887295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.683007:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508464000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39335us (39536us total) trans 4294973395 rc 0/0 00000100:00100000:1.0:1789760785.683017:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6107 00000100:00100000:1.0:1789760785.683026:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764700881473 time 764748361117 state 0x2 ref 0 00000100:00100000:2.0:1789760785.686057:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760785.686153:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508464128 00000100:00100000:3.0:1789760785.686162:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508464128 00000100:00100000:3.0:1789760785.686218:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764700881473 time 764748361117 state 0x0 ref 1 00000100:00100000:3.0:1789760785.686233:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764748621533 time 764748361117 state 0x1 ref 1 00000100:00100000:3.0:1789760785.686331:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764748621533 time 764791041098 state 0x0 ref 1 00000100:00100000:3.0:1789760785.686344:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508464128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760785.686393:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24887296, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:3.0:1789760785.686403:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56d33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.686418:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.686423:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.686429:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56d33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.686442:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56d33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.720905:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973396 is committed 00002000:00010000:1.0:1789760785.721013:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24887296 -> 24891392 00010000:00010000:1.0:1789760785.721031:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56d33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.721044:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56d33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.721054:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56d33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.721062:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760785.721067:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56d33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.721077:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56d33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.721173:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56d33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.721183:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56d33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24887296->24891391] (req 24887296->24891391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.721282:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508464128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34935us (35229us total) trans 4294973396 rc 0/0 00000100:00100000:1.0:1789760785.721332:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6108 00000100:00100000:1.0:1789760785.721343:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764748621533 time 764791041098 state 0x2 ref 0 00000100:00100000:2.0:1789760785.725259:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760785.725382:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508464256 00000100:00100000:3.0:1789760785.725391:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508464256 00000100:00100000:3.0:1789760785.725444:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764748621533 time 764791041098 state 0x0 ref 1 00000100:00100000:3.0:1789760785.725460:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764791301514 time 764791041098 state 0x1 ref 1 00000100:00100000:3.0:1789760785.725488:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764791301514 time 764830198749 state 0x0 ref 1 00000100:00100000:3.0:1789760785.725499:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508464256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760785.725544:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24891392, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:3.0:1789760785.725553:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.725569:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:3.0:1789760785.725573:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.725579:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.725593:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.762418:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973397 is committed 00002000:00010000:3.0:1789760785.762519:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24891392 -> 24895488 00010000:00010000:3.0:1789760785.762534:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.762547:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.762556:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.762564:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760785.762569:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.762579:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.762598:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.762606:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24891392->24895487] (req 24891392->24895487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.762691:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508464256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37188us (37437us total) trans 4294973397 rc 0/0 00000100:00100000:3.0:1789760785.762705:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6109 00000100:00100000:3.0:1789760785.762715:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764791301514 time 764830198749 state 0x2 ref 0 00000100:00100000:2.0:1789760785.769200:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760785.769966:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508464384 00000100:00100000:1.0:1789760785.769977:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508464384 00000100:00100000:1.0:1789760785.770037:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764791301514 time 764830198749 state 0x0 ref 1 00000100:00100000:1.0:1789760785.770053:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764830459165 time 764830198749 state 0x1 ref 1 00000100:00100000:1.0:1789760785.770076:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764830459165 time 764874787268 state 0x0 ref 1 00000100:00100000:1.0:1789760785.770100:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508464384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760785.770146:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24895488, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:1.0:1789760785.770156:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.770172:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760785.770176:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.770367:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.770385:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.811282:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973398 is committed 00002000:00010000:2.0:1789760785.811486:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24895488 -> 24899584 00010000:00010000:2.0:1789760785.811502:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.811514:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.811522:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.811529:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760785.811533:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.811540:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.811559:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.811566:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe56d41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24895488->24899583] (req 24895488->24899583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.811768:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508464384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41675us (42575us total) trans 4294973398 rc 0/0 00000100:00100000:2.0:1789760785.811779:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6110 00000100:00100000:2.0:1789760785.811788:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764830459165 time 764874787268 state 0x2 ref 0 00000100:00100000:2.0:1789760785.814953:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760785.815036:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508464512 00000100:00100000:1.0:1789760785.815046:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508464512 00000100:00100000:1.0:1789760785.815104:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764830459165 time 764874787268 state 0x0 ref 1 00000100:00100000:1.0:1789760785.815123:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764875047684 time 764874787268 state 0x1 ref 1 00000100:00100000:1.0:1789760785.815153:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764875047684 time 764919862870 state 0x0 ref 1 00000100:00100000:1.0:1789760785.815165:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508464512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760785.815219:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24899584, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:1.0:1789760785.815235:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56d48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.815249:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:1.0:1789760785.815254:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.815260:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56d48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.815274:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56d48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760785.845876:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24899584 -> 24903680 00010000:00010000:2.0:1789760785.845895:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56d48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.845908:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56d48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.845917:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56d48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.845925:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760785.845930:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56d48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.845944:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56d48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.845964:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56d48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.845972:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56d48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24899584->24903679] (req 24899584->24903679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.846066:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508464512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30898us (31118us total) trans 4294973399 rc 0/0 00000100:00100000:2.0:1789760785.846076:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6111 00000100:00100000:2.0:1789760785.846085:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764875047684 time 764919862870 state 0x2 ref 0 00000001:00080000:0.0:1789760785.846167:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973399 is committed 00000100:00100000:3.0:1789760785.850063:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760785.850268:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508464640 00000100:00100000:0.0:1789760785.850279:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508464640 00000100:00100000:0.0:1789760785.850372:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764875047684 time 764919862870 state 0x0 ref 1 00000100:00100000:0.0:1789760785.850392:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764920123286 time 764919862870 state 0x1 ref 1 00000100:00100000:0.0:1789760785.850415:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764920123286 time 764955125567 state 0x0 ref 1 00000100:00100000:0.0:1789760785.850426:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508464640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760785.850472:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24903680, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:0.0:1789760785.850482:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.850497:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:0.0:1789760785.850502:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.850509:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.850521:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.878934:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973400 is committed 00002000:00010000:0.0:1789760785.879130:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24903680 -> 24907776 00010000:00010000:0.0:1789760785.879145:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.879157:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.879166:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.879177:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760785.879182:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.879190:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.879209:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.879217:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24903680->24907775] (req 24903680->24907775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.879378:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508464640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28948us (29320us total) trans 4294973400 rc 0/0 00000100:00100000:0.0:1789760785.879389:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6112 00000100:00100000:0.0:1789760785.879398:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764920123286 time 764955125567 state 0x2 ref 0 00000100:00100000:1.0:1789760785.882810:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760785.882931:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508464768 00000100:00100000:2.0:1789760785.882940:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508464768 00000100:00100000:2.0:1789760785.882988:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764920123286 time 764955125567 state 0x0 ref 1 00000100:00100000:2.0:1789760785.883003:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764955385983 time 764955125567 state 0x1 ref 1 00000100:00100000:2.0:1789760785.883030:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764955385983 time 764987741387 state 0x0 ref 1 00000100:00100000:2.0:1789760785.883041:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508464768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760785.883083:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24907776, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:2.0:1789760785.883099:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56d56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.883115:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=764s v=(0 0) 00010000:00010000:2.0:1789760785.883119:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.883125:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56d56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.883138:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56d56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.910623:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973401 is committed 00002000:00010000:2.0:1789760785.910802:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24907776 -> 24911872 00010000:00010000:2.0:1789760785.910816:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56d56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.910828:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56d56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.910837:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56d56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.910844:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760785.910849:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56d56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.910859:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56d56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.910881:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56d56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.910890:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56d56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24907776->24911871] (req 24907776->24911871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.910985:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508464768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27940us (28178us total) trans 4294973401 rc 0/0 00000100:00100000:2.0:1789760785.910995:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6113 00000100:00100000:2.0:1789760785.911006:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764955385983 time 764987741387 state 0x2 ref 0 00000100:00100000:2.0:1789760785.914808:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760785.914902:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508464896 00000100:00100000:1.0:1789760785.914912:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508464896 00000100:00100000:1.0:1789760785.914962:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764955385983 time 764987741387 state 0x0 ref 1 00000100:00100000:1.0:1789760785.914977:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764988001803 time 764987741387 state 0x1 ref 1 00000100:00100000:1.0:1789760785.915011:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764988001803 time 765019721261 state 0x0 ref 1 00000100:00100000:1.0:1789760785.915024:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508464896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760785.915069:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24911872, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:1.0:1789760785.915077:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56d5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.915097:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760785.915103:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.915109:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56d5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.915122:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56d5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760785.947824:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24911872 -> 24915968 00010000:00010000:3.0:1789760785.947843:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56d5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.947855:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56d5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.947863:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56d5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.947870:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760785.947875:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56d5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.947884:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56d5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.947902:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56d5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.947909:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56d5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24911872->24915967] (req 24911872->24915967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.948009:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508464896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32982us (33205us total) trans 4294973402 rc 0/0 00000100:00100000:3.0:1789760785.948020:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6114 00000100:00100000:3.0:1789760785.948030:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764988001803 time 765019721261 state 0x2 ref 0 00000001:00080000:0.0:1789760785.950437:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973402 is committed 00000100:00100000:3.0:1789760785.951476:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760785.951600:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508465024 00000100:00100000:0.0:1789760785.951609:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508465024 00000100:00100000:0.0:1789760785.951714:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 764988001803 time 765019721261 state 0x0 ref 1 00000100:00100000:0.0:1789760785.951729:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765019981677 time 765019721261 state 0x1 ref 1 00000100:00100000:0.0:1789760785.951762:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765019981677 time 765056473524 state 0x0 ref 1 00000100:00100000:0.0:1789760785.951775:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508465024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760785.951818:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24915968, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:0.0:1789760785.951827:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.951842:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760785.951846:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.951851:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.951864:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760785.980882:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24915968 -> 24920064 00010000:00010000:0.0:1789760785.980906:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.980920:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.980930:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.980937:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760785.980941:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.980952:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.980974:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.980982:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24915968->24920063] (req 24915968->24920063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.981076:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508465024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29298us (29601us total) trans 4294973403 rc 0/0 00000100:00100000:0.0:1789760785.981087:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6115 00000100:00100000:0.0:1789760785.981106:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765019981677 time 765056473524 state 0x2 ref 0 00000001:00080000:0.0:1789760785.981154:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973403 is committed 00000100:00100000:3.0:1789760785.985592:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760785.985674:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508465152 00000100:00100000:2.0:1789760785.985684:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508465152 00000100:00100000:2.0:1789760785.985737:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765019981677 time 765056473524 state 0x0 ref 1 00000100:00100000:2.0:1789760785.985762:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765056733940 time 765056473524 state 0x1 ref 1 00000100:00100000:2.0:1789760785.985796:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765056733940 time 765090505858 state 0x0 ref 1 00000100:00100000:2.0:1789760785.985808:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508465152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760785.985857:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24920064, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:2.0:1789760785.985866:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56d6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.985882:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760785.985886:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.985892:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56d6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.985908:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56d6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760786.029048:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24920064 -> 24924160 00010000:00010000:2.0:1789760786.029069:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56d6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.029083:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56d6b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.029103:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56d6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.029111:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760786.029116:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56d6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.029125:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56d6b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.029145:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56d6b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.029153:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe56d6b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24920064->24924159] (req 24920064->24924159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.029245:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508465152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43434us (43658us total) trans 4294973404 rc 0/0 00000100:00100000:2.0:1789760786.029255:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6116 00000100:00100000:2.0:1789760786.029265:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765056733940 time 765090505858 state 0x2 ref 0 00000001:00080000:2.0:1789760786.029633:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973404 is committed 00000100:00100000:1.0:1789760786.032836:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760786.032933:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508465280 00000100:00100000:3.0:1789760786.032944:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508465280 00000100:00100000:3.0:1789760786.032997:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765056733940 time 765090505858 state 0x0 ref 1 00000100:00100000:3.0:1789760786.033014:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765090766274 time 765090505858 state 0x1 ref 1 00000100:00100000:3.0:1789760786.033077:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765090766274 time 765137786777 state 0x0 ref 1 00000100:00100000:3.0:1789760786.033152:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508465280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760786.033204:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24924160, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:3.0:1789760786.033214:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56d72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.033231:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760786.033235:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.033241:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56d72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.033260:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56d72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.063867:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24924160 -> 24928256 00010000:00010000:1.0:1789760786.063888:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56d72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.063902:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56d72 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.063912:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56d72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.063920:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760786.063925:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56d72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.063935:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56d72 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.063956:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56d72 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.063964:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56d72 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24924160->24928255] (req 24924160->24928255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.064063:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508465280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30971us (31231us total) trans 4294973405 rc 0/0 00000100:00100000:1.0:1789760786.064072:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6117 00000100:00100000:1.0:1789760786.064082:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765090766274 time 765137786777 state 0x2 ref 0 00000001:00080000:0.0:1789760786.064219:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973405 is committed 00000100:00100000:0.0:1789760786.068041:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760786.068194:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508465408 00000100:00100000:2.0:1789760786.068204:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508465408 00000100:00100000:2.0:1789760786.068257:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765090766274 time 765137786777 state 0x0 ref 1 00000100:00100000:2.0:1789760786.068273:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765138047193 time 765137786777 state 0x1 ref 1 00000100:00100000:2.0:1789760786.068455:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765138047193 time 765173164854 state 0x0 ref 1 00000100:00100000:2.0:1789760786.068467:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508465408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760786.068513:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24928256, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:2.0:1789760786.068522:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56d79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.068537:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760786.068542:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.068548:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56d79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.068562:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56d79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.093479:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973406 is committed 00002000:00010000:1.0:1789760786.093606:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24928256 -> 24932352 00010000:00010000:1.0:1789760786.093620:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56d79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.093633:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56d79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.093641:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56d79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.093647:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760786.093652:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56d79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.093661:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56d79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.093680:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56d79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.093687:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56d79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24928256->24932351] (req 24928256->24932351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.093895:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508465408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25423us (25856us total) trans 4294973406 rc 0/0 00000100:00100000:1.0:1789760786.093905:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6118 00000100:00100000:1.0:1789760786.093915:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765138047193 time 765173164854 state 0x2 ref 0 00000100:00100000:2.0:1789760786.096682:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760786.096879:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508465536 00000100:00100000:0.0:1789760786.096888:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508465536 00000100:00100000:0.0:1789760786.096948:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765138047193 time 765173164854 state 0x0 ref 1 00000100:00100000:0.0:1789760786.096963:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765173425270 time 765173164854 state 0x1 ref 1 00000100:00100000:0.0:1789760786.097026:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765173425270 time 765201736417 state 0x0 ref 1 00000100:00100000:0.0:1789760786.097043:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508465536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760786.097090:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24932352, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:0.0:1789760786.097098:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.097113:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760786.097117:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.097123:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.097137:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.157749:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973407 is committed 00002000:00010000:2.0:1789760786.158102:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24932352 -> 24936448 00010000:00010000:2.0:1789760786.158118:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.158131:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.158140:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.158148:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760786.158152:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.158160:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.158180:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.158187:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56d80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24932352->24936447] (req 24932352->24936447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.158282:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508465536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61237us (61606us total) trans 4294973407 rc 0/0 00000100:00100000:2.0:1789760786.158332:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6119 00000100:00100000:2.0:1789760786.158344:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765173425270 time 765201736417 state 0x2 ref 0 00000100:00100000:2.0:1789760786.162842:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760786.162981:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508465664 00000100:00100000:1.0:1789760786.162991:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508465664 00000100:00100000:1.0:1789760786.163045:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765173425270 time 765201736417 state 0x0 ref 1 00000100:00100000:1.0:1789760786.163060:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765201996833 time 765201736417 state 0x1 ref 1 00000100:00100000:1.0:1789760786.163099:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765201996833 time 765267809681 state 0x0 ref 1 00000100:00100000:1.0:1789760786.163111:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508465664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760786.163158:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24936448, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:1.0:1789760786.163167:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56d87 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.163182:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.163187:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.163193:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56d87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.163208:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56d87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.177710:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760786.197667:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973408 is committed 00002000:00010000:3.0:1789760786.197943:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24936448 -> 24940544 00010000:00010000:3.0:1789760786.197958:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56d87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.197971:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56d87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.197980:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56d87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.197987:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760786.197991:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56d87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.197999:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56d87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.198018:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56d87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.198026:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe56d87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24936448->24940543] (req 24936448->24940543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.198127:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508465664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35013us (35290us total) trans 4294973408 rc 0/0 00000100:00100000:3.0:1789760786.198138:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6120 00000100:00100000:3.0:1789760786.198147:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765201996833 time 765267809681 state 0x2 ref 0 00000100:00100000:3.0:1789760786.202274:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760786.203160:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508465792 00000100:00100000:1.0:1789760786.203172:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508465792 00000100:00100000:1.0:1789760786.203231:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765201996833 time 765267809681 state 0x0 ref 1 00000100:00100000:1.0:1789760786.203246:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765268070097 time 765267809681 state 0x1 ref 1 00000100:00100000:1.0:1789760786.203383:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765268070097 time 765308092998 state 0x0 ref 1 00000100:00100000:1.0:1789760786.203397:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508465792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760786.203446:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24940544, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:1.0:1789760786.203455:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56d8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.203470:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.203474:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.203480:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56d8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.203495:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56d8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.238595:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973409 is committed 00002000:00010000:1.0:1789760786.238616:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24940544 -> 24944640 00010000:00010000:1.0:1789760786.238632:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56d8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.238645:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56d8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.238801:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56d8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.238809:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760786.238814:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56d8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.238822:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56d8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.238842:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56d8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.238850:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56d8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24940544->24944639] (req 24940544->24944639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.238967:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508465792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35567us (36697us total) trans 4294973409 rc 0/0 00000100:00100000:1.0:1789760786.238978:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6121 00000100:00100000:1.0:1789760786.238987:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765268070097 time 765308092998 state 0x2 ref 0 00000100:00100000:2.0:1789760786.243178:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760786.243395:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508465920 00000100:00100000:0.0:1789760786.243406:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508465920 00000100:00100000:2.0:1789760786.243526:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765268070097 time 765308092998 state 0x0 ref 1 00000100:00100000:2.0:1789760786.243542:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765308353414 time 765308092998 state 0x1 ref 1 00000100:00100000:2.0:1789760786.243600:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765308353414 time 765348309947 state 0x0 ref 1 00000100:00100000:2.0:1789760786.243613:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508465920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760786.243663:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24944640, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:2.0:1789760786.243672:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56d95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.243690:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760786.243694:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.243701:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56d95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.243715:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56d95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760786.278986:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24944640 -> 24948736 00010000:00010000:0.0:1789760786.279005:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56d95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.279018:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56d95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.279028:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56d95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.279036:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760786.279041:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56d95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.279050:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56d95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.279070:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56d95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.279077:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56d95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24944640->24948735] (req 24944640->24948735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.279316:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508465920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35671us (36128us total) trans 4294973410 rc 0/0 00000100:00100000:0.0:1789760786.279338:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6122 00000100:00100000:0.0:1789760786.279348:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765308353414 time 765348309947 state 0x2 ref 0 00000001:00080000:1.0:1789760786.280048:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973410 is committed 00000100:00100000:2.0:1789760786.284383:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760786.284492:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508466048 00000100:00100000:3.0:1789760786.284502:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508466048 00000100:00100000:3.0:1789760786.284556:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765308353414 time 765348309947 state 0x0 ref 1 00000100:00100000:3.0:1789760786.284571:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765348570363 time 765348309947 state 0x1 ref 1 00000100:00100000:3.0:1789760786.284601:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765348570363 time 765389311445 state 0x0 ref 1 00000100:00100000:3.0:1789760786.284613:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508466048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760786.284658:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24948736, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:3.0:1789760786.284667:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.284683:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760786.284687:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.284694:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.284722:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760786.318802:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24948736 -> 24952832 00010000:00010000:2.0:1789760786.318824:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.318838:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.318847:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.318855:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760786.318860:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.318869:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.318891:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.318899:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56d9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24948736->24952831] (req 24948736->24952831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.318983:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508466048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34367us (34602us total) trans 4294973411 rc 0/0 00000100:00100000:2.0:1789760786.319000:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6123 00000100:00100000:2.0:1789760786.319254:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765348570363 time 765389311445 state 0x2 ref 0 00000001:00080000:3.0:1789760786.319365:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973411 is committed 00000100:00100000:2.0:1789760786.323257:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760786.323363:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508466176 00000100:00100000:1.0:1789760786.323373:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508466176 00000100:00100000:1.0:1789760786.323426:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765348570363 time 765389311445 state 0x0 ref 1 00000100:00100000:1.0:1789760786.323441:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765389571861 time 765389311445 state 0x1 ref 1 00000100:00100000:1.0:1789760786.323472:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765389571861 time 765428182134 state 0x0 ref 1 00000100:00100000:1.0:1789760786.323485:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508466176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760786.323532:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24952832, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:1.0:1789760786.323541:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56da3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.323558:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.323562:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.323568:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56da3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.323581:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56da3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.369068:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973412 is committed 00002000:00010000:1.0:1789760786.369274:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24952832 -> 24956928 00010000:00010000:1.0:1789760786.369307:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56da3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.369321:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56da3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.369330:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56da3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.369338:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760786.369343:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56da3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.369352:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56da3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.369370:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56da3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.369379:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56da3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24952832->24956927] (req 24952832->24956927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.369469:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508466176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45981us (46217us total) trans 4294973412 rc 0/0 00000100:00100000:1.0:1789760786.369483:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6124 00000100:00100000:1.0:1789760786.369492:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765389571861 time 765428182134 state 0x2 ref 0 00000100:00100000:2.0:1789760786.373358:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760786.373464:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508466304 00000100:00100000:0.0:1789760786.373474:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508466304 00000100:00100000:0.0:1789760786.373529:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765389571861 time 765428182134 state 0x0 ref 1 00000100:00100000:0.0:1789760786.373546:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765428442550 time 765428182134 state 0x1 ref 1 00000100:00100000:0.0:1789760786.373575:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765428442550 time 765478285480 state 0x0 ref 1 00000100:00100000:0.0:1789760786.373588:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508466304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760786.373751:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24956928, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:0.0:1789760786.373760:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56daa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.373775:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760786.373780:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.373785:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56daa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.373800:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56daa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.409111:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973413 is committed 00002000:00010000:3.0:1789760786.409323:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24956928 -> 24961024 00010000:00010000:3.0:1789760786.409339:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56daa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.409352:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56daa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.409360:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56daa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.409368:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760786.409373:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56daa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.409383:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56daa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.409403:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56daa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.409412:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56daa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24956928->24961023] (req 24956928->24961023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.409506:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508466304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35916us (36152us total) trans 4294973413 rc 0/0 00000100:00100000:3.0:1789760786.409517:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6125 00000100:00100000:3.0:1789760786.409526:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765428442550 time 765478285480 state 0x2 ref 0 00000100:00100000:0.0:1789760786.414330:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760786.414417:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508466432 00000100:00100000:2.0:1789760786.414426:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508466432 00000100:00100000:2.0:1789760786.414479:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765428442550 time 765478285480 state 0x0 ref 1 00000100:00100000:2.0:1789760786.414490:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765478545896 time 765478285480 state 0x1 ref 1 00000100:00100000:2.0:1789760786.414516:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765478545896 time 765519227058 state 0x0 ref 1 00000100:00100000:2.0:1789760786.414527:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508466432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760786.414574:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24961024, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:2.0:1789760786.414581:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56db1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.414593:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:2.0:1789760786.414596:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.414600:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56db1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.414610:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56db1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.443585:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24961024 -> 24965120 00010000:00010000:1.0:1789760786.443605:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56db1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.443626:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56db1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.443634:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56db1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.443640:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760786.443645:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56db1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.443653:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56db1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.443673:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56db1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.443680:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe56db1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24961024->24965119] (req 24961024->24965119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.443786:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508466432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29255us (29458us total) trans 4294973414 rc 0/0 00000100:00100000:1.0:1789760786.443797:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6126 00000100:00100000:1.0:1789760786.443806:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765478545896 time 765519227058 state 0x2 ref 0 00000100:00100000:3.0:1789760786.447002:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000001:00080000:0.0:1789760786.447112:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973414 is committed 00000100:00100000:1.0:1789760786.447531:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508466560 00000100:00100000:1.0:1789760786.447541:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508466560 00000100:00100000:1.0:1789760786.447593:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765478545896 time 765519227058 state 0x0 ref 1 00000100:00100000:1.0:1789760786.447619:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765519487474 time 765519227058 state 0x1 ref 1 00000100:00100000:1.0:1789760786.447977:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765519487474 time 765552685602 state 0x0 ref 1 00000100:00100000:1.0:1789760786.447992:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049a7d43c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508466560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760786.448038:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24965120, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:1.0:1789760786.448046:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56db8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.448062:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.448066:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.448071:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56db8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.448084:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56db8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760786.478264:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24965120 -> 24969216 00010000:00010000:1.0:1789760786.478302:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56db8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.478315:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56db8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.478324:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56db8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.478331:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760786.478335:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56db8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.478344:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56db8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.478365:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56db8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.478373:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56db8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24965120->24969215] (req 24965120->24969215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.478484:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049a7d43c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508466560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30482us (31478us total) trans 4294973415 rc 0/0 00000100:00100000:1.0:1789760786.478500:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6127 00000100:00100000:1.0:1789760786.478510:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765519487474 time 765552685602 state 0x2 ref 0 00000001:00080000:2.0:1789760786.480851:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973415 is committed 00000100:00100000:2.0:1789760786.483235:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760786.485003:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508466688 00000100:00100000:3.0:1789760786.485016:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508466688 00000100:00100000:3.0:1789760786.485079:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765519487474 time 765552685602 state 0x0 ref 1 00000100:00100000:3.0:1789760786.485164:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765552946018 time 765552685602 state 0x1 ref 1 00000100:00100000:3.0:1789760786.485230:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765552946018 time 765589940328 state 0x0 ref 1 00000100:00100000:3.0:1789760786.485245:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508466688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760786.485305:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24969216, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0xc007 00010000:00010000:3.0:1789760786.485315:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56dbf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.485331:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760786.485335:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.485342:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56dbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.485355:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56dbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.497074:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760786.498302:0:6460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508466816 00000100:00100000:1.0:1789760786.498314:0:6460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508466816 00000100:00100000:1.0:1789760786.498377:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 489 00000100:00100000:1.0:1789760786.498385:0:6460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cabc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2408:x1876699508466816:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760786.498456:0:6460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cabc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2408:x1876699508466816:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 71us (1389us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760786.498464:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 489 00000001:00080000:2.0:1789760786.510139:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973416 is committed 00002000:00010000:3.0:1789760786.510324:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24969216 -> 24973312 00002000:00010000:3.0:1789760786.510328:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760786 -> 1789760787 00002000:00010000:3.0:1789760786.510331:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760786 -> 1789760787 00010000:00010000:3.0:1789760786.510344:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56dbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.510358:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56dbf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.510367:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56dbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.510374:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760786.510378:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56dbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.510387:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56dbf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.510407:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56dbf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.510415:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56dbf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24969216->24973311] (req 24969216->24973311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.510513:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508466688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25267us (27283us total) trans 4294973416 rc 0/0 00000100:00100000:3.0:1789760786.510523:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6128 00000100:00100000:3.0:1789760786.510533:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765552946018 time 765589940328 state 0x2 ref 0 00000100:00100000:3.0:1789760786.510771:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760786.510927:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508466944 00000100:00100000:1.0:1789760786.510935:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508466944 00000100:00100000:1.0:1789760786.511003:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 979 00000100:00100000:1.0:1789760786.511014:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508466944:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760786.511068:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508466944:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 52us (294us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760786.511076:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 979 00000100:00080000:0.0:1789760786.511196:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760786.515835:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760786.515876:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760786.515918:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760786.515937:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508467200 00000100:00100000:2.0:1789760786.515949:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508467200 00000100:00100000:1.0:1789760786.515966:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508467072 00000100:00100000:1.0:1789760786.515970:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508467072 00000100:00100000:1.0:1789760786.516019:0:11662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508467328 00000100:00100000:1.0:1789760786.516022:0:11662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508467328 00000100:00100000:2.0:1789760786.516084:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 980 00000100:00100000:2.0:1789760786.516096:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508467072:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760786.516171:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508467072:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 76us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760786.516179:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 980 00000100:00100000:2.0:1789760786.516200:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 981 00000100:00100000:2.0:1789760786.516221:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508467200:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760786.516249:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508467200:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 42us (424us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760786.516255:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 981 00000100:00100000:2.0:1789760786.516269:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 982 00000100:00100000:2.0:1789760786.516276:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508467328:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760786.516336:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508467328:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 60us (417us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760786.516342:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 982 00000100:00080000:3.0:1789760786.518075:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760786.518086:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760786.521393:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760786.521451:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508467456 00000100:00100000:1.0:1789760786.521463:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508467456 00000100:00100000:1.0:1789760786.521520:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765552946018 time 765589940328 state 0x0 ref 1 00000100:00100000:1.0:1789760786.521536:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765590200744 time 765589940328 state 0x1 ref 1 00000100:00100000:1.0:1789760786.521571:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765590200744 time 765626281420 state 0x0 ref 1 00000100:00100000:1.0:1789760786.521583:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508467456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760786.521636:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24973312, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:1.0:1789760786.521645:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dc6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.521660:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.521665:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.521673:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.521686:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.554641:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973417 is committed 00002000:00010000:1.0:1789760786.554727:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24973312 -> 24977408 00010000:00010000:1.0:1789760786.554751:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.554765:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dc6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.554774:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.554781:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760786.554787:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.554796:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dc6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.554816:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dc6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.554824:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dc6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24973312->24977407] (req 24973312->24977407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.554948:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508467456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33361us (33558us total) trans 4294973417 rc 0/0 00000100:00100000:1.0:1789760786.554959:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6129 00000100:00100000:1.0:1789760786.554969:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765590200744 time 765626281420 state 0x2 ref 0 00000100:00100000:2.0:1789760786.565966:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760786.566053:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508467584 00000100:00100000:1.0:1789760786.566063:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508467584 00000100:00100000:1.0:1789760786.566476:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765590200744 time 765626281420 state 0x0 ref 1 00000100:00100000:1.0:1789760786.566495:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765626541836 time 765626281420 state 0x1 ref 1 00000100:00100000:1.0:1789760786.566516:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765626541836 time 765671227480 state 0x0 ref 1 00000100:00100000:1.0:1789760786.566530:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508467584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760786.566576:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24977408, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:1.0:1789760786.566587:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dcd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.566603:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.566607:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.566614:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.566627:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.592663:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973418 is committed 00002000:00010000:1.0:1789760786.592814:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24977408 -> 24981504 00010000:00010000:1.0:1789760786.592829:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.592842:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dcd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.592851:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dcd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.592859:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760786.592866:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dcd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.592875:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dcd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.592901:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dcd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.592907:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56dcd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24977408->24981503] (req 24977408->24981503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.593000:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508467584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26468us (27039us total) trans 4294973418 rc 0/0 00000100:00100000:1.0:1789760786.593010:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6130 00000100:00100000:1.0:1789760786.593020:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765626541836 time 765671227480 state 0x2 ref 0 00000100:00100000:3.0:1789760786.597990:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760786.598075:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508467712 00000100:00100000:0.0:1789760786.598085:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508467712 00000100:00100000:0.0:1789760786.598138:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765626541836 time 765671227480 state 0x0 ref 1 00000100:00100000:0.0:1789760786.598153:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765671487896 time 765671227480 state 0x1 ref 1 00000100:00100000:0.0:1789760786.598181:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765671487896 time 765702891576 state 0x0 ref 1 00000100:00100000:0.0:1789760786.598194:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508467712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760786.598241:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24981504, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:0.0:1789760786.598253:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56dd4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.598270:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:0.0:1789760786.598275:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.598281:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56dd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.598333:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56dd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.621796:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973419 is committed 00002000:00010000:3.0:1789760786.621858:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24981504 -> 24985600 00010000:00010000:3.0:1789760786.621873:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56dd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.621886:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56dd4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.621896:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56dd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.621903:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760786.621909:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56dd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.621918:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56dd4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.621939:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56dd4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.621945:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56dd4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24981504->24985599] (req 24981504->24985599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.622048:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508467712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23852us (24064us total) trans 4294973419 rc 0/0 00000100:00100000:3.0:1789760786.622058:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6131 00000100:00100000:3.0:1789760786.622067:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765671487896 time 765702891576 state 0x2 ref 0 00000100:00100000:2.0:1789760786.626845:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760786.628867:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508467840 00000100:00100000:1.0:1789760786.628882:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508467840 00000100:00100000:1.0:1789760786.628947:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765671487896 time 765702891576 state 0x0 ref 1 00000100:00100000:1.0:1789760786.628964:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765703151992 time 765702891576 state 0x1 ref 1 00000100:00100000:1.0:1789760786.629225:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765703151992 time 765733933753 state 0x0 ref 1 00000100:00100000:1.0:1789760786.629240:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508467840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760786.629310:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24985600, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:1.0:1789760786.629320:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56ddb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.629335:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.629340:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.629347:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56ddb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.629361:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56ddb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.658338:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973420 is committed 00002000:00010000:3.0:1789760786.658537:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24985600 -> 24989696 00010000:00010000:3.0:1789760786.658552:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56ddb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.658566:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56ddb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.658575:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56ddb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.658587:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760786.658592:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56ddb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.658601:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56ddb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.658621:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56ddb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.658629:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe56ddb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24985600->24989695] (req 24985600->24989695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.658719:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508467840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29477us (31942us total) trans 4294973420 rc 0/0 00000100:00100000:3.0:1789760786.658729:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6132 00000100:00100000:3.0:1789760786.658749:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765703151992 time 765733933753 state 0x2 ref 0 00000100:00100000:2.0:1789760786.669062:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760786.669165:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508467968 00000100:00100000:1.0:1789760786.669174:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508467968 00000100:00100000:1.0:1789760786.669229:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765703151992 time 765733933753 state 0x0 ref 1 00000100:00100000:1.0:1789760786.669244:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765734194169 time 765733933753 state 0x1 ref 1 00000100:00100000:1.0:1789760786.669363:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765734194169 time 765774071200 state 0x0 ref 1 00000100:00100000:1.0:1789760786.669376:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508467968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760786.669427:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24989696, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:1.0:1789760786.669437:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56de2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.669454:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.669458:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.669464:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56de2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.669478:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56de2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.706303:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973421 is committed 00002000:00010000:3.0:1789760786.706450:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24989696 -> 24993792 00010000:00010000:3.0:1789760786.706464:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56de2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.706475:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56de2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.706490:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56de2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.706497:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760786.706502:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56de2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.706510:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56de2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.706528:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56de2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.706534:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56de2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24989696->24993791] (req 24989696->24993791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.706668:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508467968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37289us (37611us total) trans 4294973421 rc 0/0 00000100:00100000:3.0:1789760786.706678:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6133 00000100:00100000:3.0:1789760786.706689:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765734194169 time 765774071200 state 0x2 ref 0 00000100:00100000:2.0:1789760786.710150:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760786.710763:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508468096 00000100:00100000:3.0:1789760786.710894:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508468096 00000100:00100000:3.0:1789760786.710953:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765734194169 time 765774071200 state 0x0 ref 1 00000100:00100000:3.0:1789760786.710968:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765774331616 time 765774071200 state 0x1 ref 1 00000100:00100000:3.0:1789760786.711048:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765774331616 time 765815757302 state 0x0 ref 1 00000100:00100000:3.0:1789760786.711063:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508468096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760786.711119:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24993792, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:3.0:1789760786.711129:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56de9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.711144:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760786.711148:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.711155:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56de9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.711170:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56de9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.746492:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973422 is committed 00002000:00010000:2.0:1789760786.746553:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24993792 -> 24997888 00010000:00010000:2.0:1789760786.746568:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56de9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.746580:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56de9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.746589:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56de9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.746596:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760786.746601:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56de9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.746623:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56de9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.746711:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56de9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.746719:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56de9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24993792->24997887] (req 24993792->24997887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.746935:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508468096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35869us (36790us total) trans 4294973422 rc 0/0 00000100:00100000:2.0:1789760786.746946:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6134 00000100:00100000:2.0:1789760786.746957:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765774331616 time 765815757302 state 0x2 ref 0 00000100:00100000:2.0:1789760786.750774:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760786.750901:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508468224 00000100:00100000:3.0:1789760786.750911:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508468224 00000100:00100000:3.0:1789760786.750965:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765774331616 time 765815757302 state 0x0 ref 1 00000100:00100000:3.0:1789760786.750980:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765816017718 time 765815757302 state 0x1 ref 1 00000100:00100000:3.0:1789760786.751009:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765816017718 time 765855719529 state 0x0 ref 1 00000100:00100000:3.0:1789760786.751020:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508468224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760786.751066:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 24997888, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:3.0:1789760786.751076:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56df0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.751092:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760786.751097:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.751103:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56df0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.751117:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56df0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.801276:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973423 is committed 00002000:00010000:1.0:1789760786.801401:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 24997888 -> 25001984 00010000:00010000:1.0:1789760786.801416:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56df0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.801428:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56df0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.801436:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56df0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.801443:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760786.801448:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56df0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.801456:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56df0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.801475:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56df0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.801483:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56df0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [24997888->25001983] (req 24997888->25001983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.801577:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508468224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50553us (50807us total) trans 4294973423 rc 0/0 00000100:00100000:1.0:1789760786.801591:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6135 00000100:00100000:1.0:1789760786.801601:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765816017718 time 765855719529 state 0x2 ref 0 00000100:00100000:1.0:1789760786.805404:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760786.805573:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508468352 00000100:00100000:3.0:1789760786.805583:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508468352 00000100:00100000:3.0:1789760786.805637:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765816017718 time 765855719529 state 0x0 ref 1 00000100:00100000:3.0:1789760786.805651:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765855979945 time 765855719529 state 0x1 ref 1 00000100:00100000:3.0:1789760786.805754:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765855979945 time 765910463845 state 0x0 ref 1 00000100:00100000:3.0:1789760786.805768:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508468352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760786.805815:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25001984, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:3.0:1789760786.805823:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56df7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.805838:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760786.805843:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.805849:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56df7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.805862:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56df7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.838869:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973424 is committed 00002000:00010000:3.0:1789760786.839786:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25001984 -> 25006080 00010000:00010000:3.0:1789760786.839803:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56df7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.839815:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56df7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.839824:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56df7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.839832:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760786.839837:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56df7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.839846:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56df7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.839866:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56df7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.839875:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56df7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25001984->25006079] (req 25001984->25006079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.840023:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508468352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34253us (34625us total) trans 4294973424 rc 0/0 00000100:00100000:3.0:1789760786.840034:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6136 00000100:00100000:3.0:1789760786.840044:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765855979945 time 765910463845 state 0x2 ref 0 00000100:00100000:2.0:1789760786.846048:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760786.846210:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508468480 00000100:00100000:3.0:1789760786.846219:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508468480 00000100:00100000:3.0:1789760786.846273:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765855979945 time 765910463845 state 0x0 ref 1 00000100:00100000:3.0:1789760786.846323:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765910724261 time 765910463845 state 0x1 ref 1 00000100:00100000:3.0:1789760786.846348:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765910724261 time 765951058713 state 0x0 ref 1 00000100:00100000:3.0:1789760786.846360:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508468480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760786.846407:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25006080, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:3.0:1789760786.846416:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56dfe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.846431:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:3.0:1789760786.846434:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.846441:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56dfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.846455:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56dfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.875805:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973425 is committed 00002000:00010000:2.0:1789760786.875949:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25006080 -> 25010176 00010000:00010000:2.0:1789760786.875963:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56dfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.875976:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56dfe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.875985:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56dfe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.875993:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760786.875998:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56dfe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.876007:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56dfe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.876027:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56dfe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.876035:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56dfe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25006080->25010175] (req 25006080->25010175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.876661:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508468480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30295us (30615us total) trans 4294973425 rc 0/0 00000100:00100000:2.0:1789760786.876673:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6137 00000100:00100000:2.0:1789760786.876684:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765910724261 time 765951058713 state 0x2 ref 0 00000100:00100000:2.0:1789760786.884155:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760786.884410:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508468608 00000100:00100000:1.0:1789760786.884420:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508468608 00000100:00100000:1.0:1789760786.884474:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765910724261 time 765951058713 state 0x0 ref 1 00000100:00100000:1.0:1789760786.884490:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765951319129 time 765951058713 state 0x1 ref 1 00000100:00100000:1.0:1789760786.884520:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765951319129 time 765989230352 state 0x0 ref 1 00000100:00100000:1.0:1789760786.884532:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508468608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760786.884581:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25010176, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:1.0:1789760786.884590:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56e05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.884605:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=765s v=(0 0) 00010000:00010000:1.0:1789760786.884609:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.884616:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56e05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.884630:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56e05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.912751:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973426 is committed 00002000:00010000:3.0:1789760786.913133:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25010176 -> 25014272 00010000:00010000:3.0:1789760786.913149:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56e05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.913163:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56e05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.913173:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56e05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.913181:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760786.913186:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56e05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.913197:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56e05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.913219:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56e05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.913227:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56e05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25010176->25014271] (req 25010176->25014271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.913409:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508468608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28873us (29259us total) trans 4294973426 rc 0/0 00000100:00100000:3.0:1789760786.913425:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6138 00000100:00100000:3.0:1789760786.913435:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765951319129 time 765989230352 state 0x2 ref 0 00000100:00100000:1.0:1789760786.919053:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760786.919170:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508468736 00000100:00100000:3.0:1789760786.919180:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508468736 00000100:00100000:3.0:1789760786.919233:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765951319129 time 765989230352 state 0x0 ref 1 00000100:00100000:3.0:1789760786.919248:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765989490768 time 765989230352 state 0x1 ref 1 00000100:00100000:3.0:1789760786.919315:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765989490768 time 766023995162 state 0x0 ref 1 00000100:00100000:3.0:1789760786.919329:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508468736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760786.919376:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25014272, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:3.0:1789760786.919385:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.919401:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760786.919406:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.919412:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.919426:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760786.949431:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25014272 -> 25018368 00010000:00010000:0.0:1789760786.949449:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.949463:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.949472:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.949480:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760786.949484:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.949502:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.949522:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.949530:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25014272->25018367] (req 25014272->25018367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.949627:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508468736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30296us (30578us total) trans 4294973427 rc 0/0 00000100:00100000:0.0:1789760786.949637:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6139 00000100:00100000:0.0:1789760786.949658:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765989490768 time 766023995162 state 0x2 ref 0 00000001:00080000:2.0:1789760786.951315:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973427 is committed 00000100:00100000:0.0:1789760786.957519:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760786.957687:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508468864 00000100:00100000:2.0:1789760786.957698:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508468864 00000100:00100000:2.0:1789760786.957765:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 765989490768 time 766023995162 state 0x0 ref 1 00000100:00100000:2.0:1789760786.957782:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766024255578 time 766023995162 state 0x1 ref 1 00000100:00100000:2.0:1789760786.957891:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766024255578 time 766062600966 state 0x0 ref 1 00000100:00100000:2.0:1789760786.957906:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508468864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760786.957961:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25018368, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:2.0:1789760786.957971:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.957988:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760786.957992:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.957999:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.958012:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.008403:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973428 is committed 00002000:00010000:3.0:1789760787.008485:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25018368 -> 25022464 00010000:00010000:3.0:1789760787.008500:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.008514:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.008524:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.008532:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760787.008537:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.008546:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.008565:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.008572:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25018368->25022463] (req 25018368->25022463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.008772:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508468864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50863us (51256us total) trans 4294973428 rc 0/0 00000100:00100000:3.0:1789760787.008845:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6140 00000100:00100000:3.0:1789760787.008858:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766024255578 time 766062600966 state 0x2 ref 0 00000100:00100000:3.0:1789760787.013073:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760787.013594:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508468992 00000100:00100000:1.0:1789760787.013605:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508468992 00000100:00100000:1.0:1789760787.013689:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766024255578 time 766062600966 state 0x0 ref 1 00000100:00100000:1.0:1789760787.013704:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766062861382 time 766062600966 state 0x1 ref 1 00000100:00100000:1.0:1789760787.013802:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766062861382 time 766118510836 state 0x0 ref 1 00000100:00100000:1.0:1789760787.013827:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508468992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760787.013876:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25022464, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:1.0:1789760787.013886:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56e1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.013902:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:1.0:1789760787.013906:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.013913:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56e1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.013928:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56e1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760787.014516:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789760787.040277:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25022464 -> 25026560 00010000:00010000:2.0:1789760787.040316:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56e1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.040329:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56e1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.040338:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56e1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.040344:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760787.040349:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56e1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.040357:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56e1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.040376:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56e1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.040383:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe56e1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25022464->25026559] (req 25022464->25026559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.040478:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508468992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26660us (27411us total) trans 4294973429 rc 0/0 00000100:00100000:2.0:1789760787.040490:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6141 00000100:00100000:2.0:1789760787.040500:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766062861382 time 766118510836 state 0x2 ref 0 00000001:00080000:0.0:1789760787.041048:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973429 is committed 00000100:00100000:1.0:1789760787.048572:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760787.048664:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508469120 00000100:00100000:0.0:1789760787.048674:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508469120 00000100:00100000:0.0:1789760787.048728:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766062861382 time 766118510836 state 0x0 ref 1 00000100:00100000:0.0:1789760787.048758:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766118771252 time 766118510836 state 0x1 ref 1 00000100:00100000:2.0:1789760787.048886:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766118771252 time 766153594050 state 0x0 ref 1 00000100:00100000:2.0:1789760787.048905:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508469120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760787.048955:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25026560, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:2.0:1789760787.048963:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.048980:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.048984:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.048990:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.049004:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.085550:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973430 is committed 00002000:00010000:0.0:1789760787.085614:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25026560 -> 25030656 00010000:00010000:0.0:1789760787.085628:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.085639:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.085649:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.085656:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760787.085661:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.085670:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.085689:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.085696:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe56e21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25026560->25030655] (req 25026560->25030655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.086007:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508469120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37100us (37440us total) trans 4294973430 rc 0/0 00000100:00100000:0.0:1789760787.086018:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6142 00000100:00100000:0.0:1789760787.086028:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766118771252 time 766153594050 state 0x2 ref 0 00000100:00100000:3.0:1789760787.092397:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760787.092549:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508469248 00000100:00100000:2.0:1789760787.092560:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508469248 00000100:00100000:2.0:1789760787.092613:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766118771252 time 766153594050 state 0x0 ref 1 00000100:00100000:2.0:1789760787.092630:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766153854466 time 766153594050 state 0x1 ref 1 00000100:00100000:2.0:1789760787.092667:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766153854466 time 766197376901 state 0x0 ref 1 00000100:00100000:2.0:1789760787.092682:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508469248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760787.092742:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25030656, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:2.0:1789760787.092751:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.092766:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.092771:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.092778:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.092792:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.120506:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973431 is committed 00002000:00010000:2.0:1789760787.120757:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25030656 -> 25034752 00010000:00010000:2.0:1789760787.120773:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.120787:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.120801:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.120808:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760787.120813:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.120822:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.120842:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.120850:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25030656->25034751] (req 25030656->25034751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.120978:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508469248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28294us (28588us total) trans 4294973431 rc 0/0 00000100:00100000:2.0:1789760787.120989:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6143 00000100:00100000:2.0:1789760787.120999:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766153854466 time 766197376901 state 0x2 ref 0 00000100:00100000:3.0:1789760787.128247:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760787.128906:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508469376 00000100:00100000:2.0:1789760787.128918:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508469376 00000100:00100000:2.0:1789760787.128971:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766153854466 time 766197376901 state 0x0 ref 1 00000100:00100000:2.0:1789760787.128986:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766197637317 time 766197376901 state 0x1 ref 1 00000100:00100000:2.0:1789760787.129066:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766197637317 time 766233776529 state 0x0 ref 1 00000100:00100000:2.0:1789760787.129079:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508469376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760787.129138:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25034752, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:2.0:1789760787.129147:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.129163:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.129167:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.129173:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.129187:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760787.177060:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25034752 -> 25038848 00010000:00010000:3.0:1789760787.177077:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.177103:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.177112:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.177119:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760787.177124:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.177132:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.177162:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.177170:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe56e2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25034752->25038847] (req 25034752->25038847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.177272:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508469376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48190us (49028us total) trans 4294973432 rc 0/0 00000100:00100000:3.0:1789760787.177316:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6144 00000100:00100000:3.0:1789760787.177326:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766197637317 time 766233776529 state 0x2 ref 0 00000001:00080000:0.0:1789760787.177676:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973432 is committed 00000100:00100000:1.0:1789760787.182865:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760787.183239:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508469504 00000100:00100000:0.0:1789760787.183249:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508469504 00000100:00100000:0.0:1789760787.183322:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766197637317 time 766233776529 state 0x0 ref 1 00000100:00100000:0.0:1789760787.183337:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766234036945 time 766233776529 state 0x1 ref 1 00000100:00100000:0.0:1789760787.183356:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766234036945 time 766288067314 state 0x0 ref 1 00000100:00100000:0.0:1789760787.183369:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508469504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760787.183418:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25038848, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:0.0:1789760787.183427:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56e36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.183442:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760787.183446:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.183453:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56e36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.183467:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56e36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.201589:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760787.214259:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973433 is committed 00002000:00010000:2.0:1789760787.214572:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25038848 -> 25042944 00010000:00010000:2.0:1789760787.214587:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56e36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.214600:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56e36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.214608:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56e36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.214615:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760787.214620:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56e36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.214629:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56e36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.214658:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56e36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.214667:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56e36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25038848->25042943] (req 25038848->25042943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.214765:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508469504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31394us (31905us total) trans 4294973433 rc 0/0 00000100:00100000:2.0:1789760787.214783:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6145 00000100:00100000:2.0:1789760787.214793:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766234036945 time 766288067314 state 0x2 ref 0 00000100:00100000:2.0:1789760787.220105:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760787.220233:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508469632 00000100:00100000:0.0:1789760787.220243:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508469632 00000100:00100000:0.0:1789760787.220435:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766234036945 time 766288067314 state 0x0 ref 1 00000100:00100000:0.0:1789760787.220451:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766288327730 time 766288067314 state 0x1 ref 1 00000100:00100000:0.0:1789760787.220530:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766288327730 time 766325238839 state 0x0 ref 1 00000100:00100000:0.0:1789760787.220544:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508469632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760787.220597:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25042944, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:0.0:1789760787.220606:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.220621:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760787.220626:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.220632:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.220646:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.253514:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973434 is committed 00002000:00010000:2.0:1789760787.257013:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25042944 -> 25047040 00010000:00010000:2.0:1789760787.257034:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.257047:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.257056:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.257063:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760787.257068:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.257077:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.257102:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.257110:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25042944->25047039] (req 25042944->25047039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.257215:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508469632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36668us (37114us total) trans 4294973434 rc 0/0 00000100:00100000:2.0:1789760787.257226:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6146 00000100:00100000:2.0:1789760787.257235:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766288327730 time 766325238839 state 0x2 ref 0 00000100:00100000:3.0:1789760787.262004:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760787.262068:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508469760 00000100:00100000:2.0:1789760787.262079:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508469760 00000100:00100000:2.0:1789760787.262146:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766288327730 time 766325238839 state 0x0 ref 1 00000100:00100000:2.0:1789760787.262162:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766325499255 time 766325238839 state 0x1 ref 1 00000100:00100000:2.0:1789760787.262192:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766325499255 time 766366902660 state 0x0 ref 1 00000100:00100000:2.0:1789760787.262206:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508469760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760787.262256:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25047040, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:2.0:1789760787.262265:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56e44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.262281:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.262320:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.262327:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56e44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.262342:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56e44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.303357:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973435 is committed 00002000:00010000:3.0:1789760787.303422:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25047040 -> 25051136 00010000:00010000:3.0:1789760787.303438:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56e44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.303451:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56e44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.303460:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56e44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.303467:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760787.303472:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56e44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.303480:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56e44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.303498:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56e44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.303509:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56e44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25047040->25051135] (req 25047040->25051135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.303598:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508469760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41391us (41599us total) trans 4294973435 rc 0/0 00000100:00100000:3.0:1789760787.303609:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6147 00000100:00100000:3.0:1789760787.303618:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766325499255 time 766366902660 state 0x2 ref 0 00000100:00100000:1.0:1789760787.308156:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760787.310232:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508469888 00000100:00100000:2.0:1789760787.310248:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508469888 00000100:00100000:2.0:1789760787.310335:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766325499255 time 766366902660 state 0x0 ref 1 00000100:00100000:2.0:1789760787.310361:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766367163076 time 766366902660 state 0x1 ref 1 00000100:00100000:2.0:1789760787.310396:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766367163076 time 766415106085 state 0x0 ref 1 00000100:00100000:2.0:1789760787.310410:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508469888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760787.310461:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25051136, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:2.0:1789760787.310470:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56e4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.310485:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.310490:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.310496:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56e4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.310511:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56e4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.327202:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3474:1876699534089088:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.327276:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.327353:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534089600:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.327402:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.327425:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534090112:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.327450:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.327469:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534090624:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.327490:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.327515:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:3.0:1789760787.327521:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534089216:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.327548:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.327566:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534089728:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.327583:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.327599:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534090240:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.327619:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.327638:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534090752:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760787.327663:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.327706:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534089088 00000100:00100000:3.0:1789760787.327713:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534089088 00000100:00100000:3.0:1789760787.327768:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 490 00000100:00100000:3.0:1789760787.327777:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988ca9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3474:x1876699534089088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.328270:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988ca9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3474:x1876699534089088:12345-0@lo:400:kworker.0 Request processed in 492us (992us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.328280:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 490 00000100:00100000:3.0:1789760787.328333:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534089600 00000100:00100000:3.0:1789760787.328338:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534089600 00000100:00100000:1.0:1789760787.328494:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534088960:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760787.328527:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.328551:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f30380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3473:1876699534089472:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.328572:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.328587:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534089984:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.328607:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.328623:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534090496:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.328643:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.328663:0:3473:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1789760787.328668:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fcdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534089344:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.328691:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.328708:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3473:1876699534089856:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.328741:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.328759:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887f4d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3473:1876699534090368:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.328780:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.328938:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534090752 00000100:00100000:1.0:1789760787.328942:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534090752 00000100:00100000:1.0:1789760787.328984:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534088960 00000100:00100000:1.0:1789760787.328988:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534088960 00000100:00100000:1.0:1789760787.329006:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:1.0:1789760787.329013:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534090752:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760787.329044:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789760787.329183:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534089344 00000100:00100000:3.0:1789760787.329186:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534089344 00000100:00100000:3.0:1789760787.329201:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 466 00000100:00100000:3.0:1789760787.329206:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534089600:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.329245:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534089600:12345-0@lo:400:kworker.0 Request processed in 38us (1843us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.329251:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 466 00000100:00100000:3.0:1789760787.329265:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:0.0:1789760787.329386:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534089216 00000100:00100000:0.0:1789760787.329390:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534089216 00000100:00100000:0.0:1789760787.329411:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534089728 00000100:00100000:0.0:1789760787.329414:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534089728 00000100:00100000:0.0:1789760787.329426:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534090240 00000100:00100000:0.0:1789760787.329428:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534090240 00000100:00100000:0.0:1789760787.329440:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534089472 00000100:00100000:0.0:1789760787.329442:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534089472 00000100:00100000:0.0:1789760787.329452:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534089856 00000100:00100000:0.0:1789760787.329454:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534089856 00000100:00100000:0.0:1789760787.329465:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534090368 00000100:00100000:0.0:1789760787.329467:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534090368 00000100:00100000:0.0:1789760787.329480:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 983 00000100:00100000:0.0:1789760787.329486:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534089216:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.329521:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534089216:12345-0@lo:400:kworker.0 Request processed in 36us (1972us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.329528:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 983 00000100:00100000:0.0:1789760787.329540:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 984 00000100:00100000:0.0:1789760787.329546:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534089728:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.329574:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534089728:12345-0@lo:400:kworker.0 Request processed in 28us (1989us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.329580:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 984 00000100:00100000:0.0:1789760787.329591:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 985 00000100:00100000:0.0:1789760787.329597:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534090240:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.329623:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534090240:12345-0@lo:400:kworker.0 Request processed in 27us (2003us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.329629:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 985 00000100:00100000:0.0:1789760787.329641:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 986 00000100:00100000:0.0:1789760787.329647:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3473:x1876699534089472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.329673:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3473:x1876699534089472:12345-0@lo:400:kworker.0 Request processed in 27us (1100us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.329679:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 986 00000100:00100000:0.0:1789760787.329689:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 987 00000100:00100000:0.0:1789760787.329695:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3473:x1876699534089856:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.329722:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3473:x1876699534089856:12345-0@lo:400:kworker.0 Request processed in 27us (991us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.329728:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 987 00000100:00100000:0.0:1789760787.329749:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 988 00000100:00100000:0.0:1789760787.329755:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3473:x1876699534090368:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.329785:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3473:x1876699534090368:12345-0@lo:400:kworker.0 Request processed in 30us (1004us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.329790:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 988 00000100:00100000:3.0:1789760787.329997:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3473:x1876699534089344:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.330033:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3473:x1876699534089344:12345-0@lo:400:kworker.0 Request processed in 763us (1340us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.330040:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:3.0:1789760787.330062:0:8103:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534090112 00000100:00100000:3.0:1789760787.330066:0:8103:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534090112 00000100:00100000:1.0:1789760787.330177:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3474:x1876699534090752:12345-0@lo:13:osp-pre-1-1.0 Request processed in 68us (1416us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760787.330185:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00100000:1.0:1789760787.330204:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:1.0:1789760787.330211:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534088960:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760787.330224:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760787.330254:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534088960:12345-0@lo:13:osp-pre-1-0.0 Request processed in 45us (1727us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760787.330261:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:1.0:1789760787.330332:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7342680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534090880:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760787.330441:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534090624 00000100:00100000:0.0:1789760787.330446:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534090624 00000100:00100000:0.0:1789760787.330465:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534089984 00000100:00100000:0.0:1789760787.330468:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534089984 00000100:00100000:0.0:1789760787.330481:0:8105:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534090496 00000100:00100000:0.0:1789760787.330484:0:8105:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534090496 00000100:00100000:0.0:1789760787.330495:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 248 00000100:00100000:0.0:1789760787.330500:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534090624:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.330533:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534090624:12345-0@lo:400:kworker.0 Request processed in 32us (3042us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.330538:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 248 00000100:00100000:0.0:1789760787.330548:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 249 00000100:00100000:0.0:1789760787.330554:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534089984:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.330581:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534089984:12345-0@lo:400:kworker.0 Request processed in 27us (1973us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.330586:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 249 00000100:00100000:0.0:1789760787.330597:0:8105:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 250 00000100:00100000:0.0:1789760787.330602:0:8105:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534090496:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.330627:0:8105:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7342300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3473:x1876699534090496:12345-0@lo:400:kworker.0 Request processed in 25us (1982us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.330632:0:8105:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1789760787.330910:0:8103:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 251 00000100:00100000:3.0:1789760787.330919:0:8103:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534090112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.330958:0:8103:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534090112:12345-0@lo:400:kworker.0 Request processed in 41us (3506us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.330967:0:8103:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 251 00000100:00080000:3.0:1789760787.331005:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789760787.331010:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760787.331012:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760787.331014:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760787.331157:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.331193:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789760787.331201:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499c2a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534091008:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760787.331224:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.331254:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534090880 00000100:00100000:1.0:1789760787.331258:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534090880 00000100:00100000:1.0:1789760787.331277:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534091008 00000100:00100000:1.0:1789760787.331280:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534091008 00000100:00100000:1.0:1789760787.331314:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:1.0:1789760787.331321:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534090880:12345-0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760787.331485:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534089216:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.331501:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534089728:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.331513:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534090240:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.331530:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534090752:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760787.331541:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3474:1876699534089088:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.331552:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534089600:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.331561:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534090112:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.331572:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534090624:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.331763:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:3.0:1789760787.331773:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534091008:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760787.331789:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527499264 pending 1703936 free 3833593856 avail 3822059520 00000100:00100000:3.0:1789760787.331845:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7343800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534091008:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (620us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.331852:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00002000:00000024:1.0:1789760787.331969:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527499264 pending 1703936 free 3833593856 avail 3822059520 00000100:00100000:1.0:1789760787.331998:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7341500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534090880:12345-0@lo:13:osp-pre-0-1.0 Request processed in 678us (842us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760787.332004:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:1.0:1789760787.332258:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fcdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534089344:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.332271:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3473:1876699534089856:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.332307:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3473:1876699534090368:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.332326:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534088960:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760787.332337:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f30380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3473:1876699534089472:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.332346:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534089984:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.332354:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887f4fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3473:1876699534090496:0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.333349:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499c2a4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534091008:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760787.333367:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7342680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534090880:0@lo:13:osp-pre-0-1.0 00000001:00080000:2.0:1789760787.367491:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973436 is committed 00002000:00010000:3.0:1789760787.367641:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25051136 -> 25055232 00010000:00010000:3.0:1789760787.367657:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56e4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.367669:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56e4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.367677:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56e4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.367684:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760787.367689:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56e4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.367703:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56e4b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.367722:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56e4b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.367739:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56e4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25051136->25055231] (req 25051136->25055231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.367829:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508469888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57417us (59679us total) trans 4294973436 rc 0/0 00000100:00100000:3.0:1789760787.367840:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6148 00000100:00100000:3.0:1789760787.367850:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766367163076 time 766415106085 state 0x2 ref 0 00000100:00100000:2.0:1789760787.371495:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760787.371578:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508470016 00000100:00100000:0.0:1789760787.371587:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508470016 00000100:00100000:0.0:1789760787.371637:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766367163076 time 766415106085 state 0x0 ref 1 00000100:00100000:0.0:1789760787.371651:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766415366501 time 766415106085 state 0x1 ref 1 00000100:00100000:0.0:1789760787.371689:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766415366501 time 766476399427 state 0x0 ref 1 00000100:00100000:0.0:1789760787.371702:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508470016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760787.371761:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25055232, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:0.0:1789760787.371770:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.371786:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760787.371790:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.371796:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.371810:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.418681:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973437 is committed 00002000:00010000:2.0:1789760787.418700:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25055232 -> 25059328 00010000:00010000:2.0:1789760787.418715:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.418728:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.418746:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.418754:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760787.418759:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.418774:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.418794:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.418803:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56e52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25055232->25059327] (req 25055232->25059327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.418898:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508470016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47193us (47406us total) trans 4294973437 rc 0/0 00000100:00100000:2.0:1789760787.418907:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6149 00000100:00100000:2.0:1789760787.418916:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766415366501 time 766476399427 state 0x2 ref 0 00000100:00100000:2.0:1789760787.426514:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760787.426576:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508470144 00000100:00100000:3.0:1789760787.426590:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508470144 00000100:00100000:3.0:1789760787.426646:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766415366501 time 766476399427 state 0x0 ref 1 00000100:00100000:3.0:1789760787.426665:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766476659843 time 766476399427 state 0x1 ref 1 00000100:00100000:3.0:1789760787.426752:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766476659843 time 766531462396 state 0x0 ref 1 00000100:00100000:3.0:1789760787.426861:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508470144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760787.426913:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25059328, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:3.0:1789760787.426923:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.426938:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760787.426942:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.426949:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.426962:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.467575:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973438 is committed 00002000:00010000:3.0:1789760787.468041:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25059328 -> 25063424 00010000:00010000:3.0:1789760787.468057:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.468070:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.468089:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.468097:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760787.468102:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.468111:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.468131:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e59 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.468140:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25059328->25063423] (req 25059328->25063423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.468382:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508470144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41614us (41873us total) trans 4294973438 rc 0/0 00000100:00100000:3.0:1789760787.468393:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6150 00000100:00100000:3.0:1789760787.468403:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766476659843 time 766531462396 state 0x2 ref 0 00000100:00100000:0.0:1789760787.486516:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760787.486572:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508470272 00000100:00100000:3.0:1789760787.486582:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508470272 00000100:00100000:3.0:1789760787.486643:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766476659843 time 766531462396 state 0x0 ref 1 00000100:00100000:3.0:1789760787.486658:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766531722812 time 766531462396 state 0x1 ref 1 00000100:00100000:3.0:1789760787.486689:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766531722812 time 766591399436 state 0x0 ref 1 00000100:00100000:3.0:1789760787.486703:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508470272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760787.486760:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25063424, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0xc007 00010000:00010000:3.0:1789760787.486778:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.486793:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760787.486797:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.486804:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.486823:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.535846:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973439 is committed 00002000:00010000:3.0:1789760787.536791:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25063424 -> 25067520 00002000:00010000:3.0:1789760787.536798:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760787 -> 1789760788 00002000:00010000:3.0:1789760787.536800:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760787 -> 1789760788 00010000:00010000:3.0:1789760787.536813:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.536825:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.536834:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.536840:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760787.536845:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.536854:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.536873:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.536886:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25063424->25067519] (req 25063424->25067519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.536983:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508470272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50277us (50471us total) trans 4294973439 rc 0/0 00000100:00100000:3.0:1789760787.536993:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6151 00000100:00100000:3.0:1789760787.537002:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766531722812 time 766591399436 state 0x2 ref 0 00000100:00100000:0.0:1789760787.540244:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760787.540355:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508470400 00000100:00100000:2.0:1789760787.540364:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508470400 00000100:00100000:2.0:1789760787.540417:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766531722812 time 766591399436 state 0x0 ref 1 00000100:00100000:2.0:1789760787.540432:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766591659852 time 766591399436 state 0x1 ref 1 00000100:00100000:2.0:1789760787.540463:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766591659852 time 766645173683 state 0x0 ref 1 00000100:00100000:2.0:1789760787.540475:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508470400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760787.540523:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25067520, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:2.0:1789760787.540532:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56e67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.540546:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.540550:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.540556:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56e67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.540569:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56e67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760787.611828:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25067520 -> 25071616 00010000:00010000:0.0:1789760787.611849:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56e67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.611862:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56e67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.611872:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56e67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.611885:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760787.611890:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56e67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.611900:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56e67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.611929:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56e67 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.611938:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe56e67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25067520->25071615] (req 25067520->25071615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.612036:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508470400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 71558us (71797us total) trans 4294973440 rc 0/0 00000100:00100000:0.0:1789760787.612047:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6152 00000100:00100000:0.0:1789760787.612056:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766591659852 time 766645173683 state 0x2 ref 0 00000001:00080000:2.0:1789760787.612493:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973440 is committed 00000100:00100000:2.0:1789760787.615531:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760787.615641:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508470528 00000100:00100000:3.0:1789760787.615660:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508470528 00000100:00100000:3.0:1789760787.615716:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766591659852 time 766645173683 state 0x0 ref 1 00000100:00100000:3.0:1789760787.615742:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766645434099 time 766645173683 state 0x1 ref 1 00000100:00100000:3.0:1789760787.615773:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766645434099 time 766720483412 state 0x0 ref 1 00000100:00100000:3.0:1789760787.615786:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508470528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760787.615833:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25071616, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:3.0:1789760787.615842:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e6e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.615859:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760787.615864:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.615870:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.615888:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.662589:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973441 is committed 00002000:00010000:0.0:1789760787.662696:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25071616 -> 25075712 00010000:00010000:0.0:1789760787.662711:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.662724:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.662748:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.662755:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760787.662824:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.662835:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.662854:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.662861:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe56e6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25071616->25075711] (req 25071616->25075711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.662987:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508470528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47198us (47460us total) trans 4294973441 rc 0/0 00000100:00100000:0.0:1789760787.662997:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6153 00000100:00100000:0.0:1789760787.663006:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766645434099 time 766720483412 state 0x2 ref 0 00000100:00100000:3.0:1789760787.668516:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760787.668624:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508470656 00000100:00100000:2.0:1789760787.668634:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508470656 00000100:00100000:2.0:1789760787.668687:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766645434099 time 766720483412 state 0x0 ref 1 00000100:00100000:2.0:1789760787.668701:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766720743828 time 766720483412 state 0x1 ref 1 00000100:00100000:2.0:1789760787.668735:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766720743828 time 766773437739 state 0x0 ref 1 00000100:00100000:2.0:1789760787.668748:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508470656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760787.668821:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25075712, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:2.0:1789760787.668831:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56e75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.668847:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.668852:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.668858:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56e75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.668873:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56e75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.728515:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973442 is committed 00002000:00010000:3.0:1789760787.728975:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25075712 -> 25079808 00010000:00010000:3.0:1789760787.728992:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56e75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.729004:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56e75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.729013:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56e75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.729020:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760787.729025:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56e75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.729034:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56e75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.729053:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56e75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.729060:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe56e75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25075712->25079807] (req 25075712->25079807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.729210:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508470656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60459us (60700us total) trans 4294973442 rc 0/0 00000100:00100000:3.0:1789760787.729221:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6154 00000100:00100000:3.0:1789760787.730779:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766720743828 time 766773437739 state 0x2 ref 0 00000100:00100000:2.0:1789760787.734838:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760787.734921:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508470784 00000100:00100000:0.0:1789760787.734930:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508470784 00000100:00100000:0.0:1789760787.734983:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766720743828 time 766773437739 state 0x0 ref 1 00000100:00100000:0.0:1789760787.734999:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766773698155 time 766773437739 state 0x1 ref 1 00000100:00100000:0.0:1789760787.735031:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766773698155 time 766839741846 state 0x0 ref 1 00000100:00100000:0.0:1789760787.735042:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508470784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760787.735095:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25079808, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:0.0:1789760787.735104:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56e7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.735120:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760787.735124:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.735131:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56e7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.735146:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56e7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760787.777509:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25079808 -> 25083904 00010000:00010000:0.0:1789760787.777529:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56e7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.777543:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56e7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.777553:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56e7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.777562:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760787.777569:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56e7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.777578:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56e7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.777598:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56e7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.777607:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56e7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25079808->25083903] (req 25079808->25083903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.777707:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508470784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42662us (42874us total) trans 4294973443 rc 0/0 00000100:00100000:0.0:1789760787.777717:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6155 00000100:00100000:0.0:1789760787.777737:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766773698155 time 766839741846 state 0x2 ref 0 00000001:00080000:1.0:1789760787.778519:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973443 is committed 00000100:00100000:2.0:1789760787.782828:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760787.782905:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508470912 00000100:00100000:0.0:1789760787.782914:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508470912 00000100:00100000:0.0:1789760787.782969:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766773698155 time 766839741846 state 0x0 ref 1 00000100:00100000:0.0:1789760787.782983:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766840002262 time 766839741846 state 0x1 ref 1 00000100:00100000:0.0:1789760787.783025:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766840002262 time 766887734109 state 0x0 ref 1 00000100:00100000:0.0:1789760787.783038:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508470912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760787.783183:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25083904, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:0.0:1789760787.783193:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56e83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.783208:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:0.0:1789760787.783213:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.783219:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56e83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.783232:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56e83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.817308:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973444 is committed 00002000:00010000:2.0:1789760787.821267:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25083904 -> 25088000 00010000:00010000:2.0:1789760787.821304:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56e83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.821318:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56e83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.821327:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56e83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.821334:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760787.821339:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56e83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.821349:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56e83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.821369:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56e83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.821378:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe56e83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25083904->25087999] (req 25083904->25087999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.821469:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508470912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38427us (38711us total) trans 4294973444 rc 0/0 00000100:00100000:2.0:1789760787.821479:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6156 00000100:00100000:2.0:1789760787.821489:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766840002262 time 766887734109 state 0x2 ref 0 00000100:00100000:2.0:1789760787.830686:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760787.830818:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508471040 00000100:00100000:3.0:1789760787.830829:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508471040 00000100:00100000:3.0:1789760787.830880:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766840002262 time 766887734109 state 0x0 ref 1 00000100:00100000:3.0:1789760787.830894:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766887994525 time 766887734109 state 0x1 ref 1 00000100:00100000:3.0:1789760787.830924:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766887994525 time 766935635428 state 0x0 ref 1 00000100:00100000:3.0:1789760787.830935:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508471040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760787.830979:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25088000, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:3.0:1789760787.830988:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.831003:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:3.0:1789760787.831006:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.831012:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.831026:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.873412:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973445 is committed 00002000:00010000:3.0:1789760787.873526:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25088000 -> 25092096 00010000:00010000:3.0:1789760787.873541:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.873555:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.873565:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.873573:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760787.873577:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.873587:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.873609:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.873616:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56e8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25088000->25092095] (req 25088000->25092095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.874041:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508471040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43101us (43359us total) trans 4294973445 rc 0/0 00000100:00100000:3.0:1789760787.874056:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6157 00000100:00100000:3.0:1789760787.874066:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766887994525 time 766935635428 state 0x2 ref 0 00000100:00100000:3.0:1789760787.881870:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760787.881973:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508471168 00000100:00100000:2.0:1789760787.881983:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508471168 00000100:00100000:2.0:1789760787.882038:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766887994525 time 766935635428 state 0x0 ref 1 00000100:00100000:2.0:1789760787.882054:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766935895844 time 766935635428 state 0x1 ref 1 00000100:00100000:2.0:1789760787.882093:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766935895844 time 766986803303 state 0x0 ref 1 00000100:00100000:2.0:1789760787.882105:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508471168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760787.882154:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25092096, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:2.0:1789760787.882164:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56e91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.882180:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=766s v=(0 0) 00010000:00010000:2.0:1789760787.882185:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.882192:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56e91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.882206:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56e91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.910940:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973446 is committed 00002000:00010000:2.0:1789760787.911089:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25092096 -> 25096192 00010000:00010000:2.0:1789760787.911105:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56e91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.911117:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56e91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.911125:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56e91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.911132:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760787.911137:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56e91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.911146:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56e91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.911174:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56e91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.911183:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe56e91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25092096->25096191] (req 25092096->25096191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.911280:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508471168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29171us (29414us total) trans 4294973446 rc 0/0 00000100:00100000:2.0:1789760787.911524:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6158 00000100:00100000:2.0:1789760787.911535:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766935895844 time 766986803303 state 0x2 ref 0 00000100:00100000:0.0:1789760787.915696:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760787.915855:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508471296 00000100:00100000:2.0:1789760787.915865:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508471296 00000100:00100000:2.0:1789760787.915918:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766935895844 time 766986803303 state 0x0 ref 1 00000100:00100000:2.0:1789760787.915934:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766987063719 time 766986803303 state 0x1 ref 1 00000100:00100000:2.0:1789760787.915997:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766987063719 time 767020706704 state 0x0 ref 1 00000100:00100000:2.0:1789760787.916010:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508471296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760787.916056:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25096192, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:2.0:1789760787.916065:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56e98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.916141:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760787.916147:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.916154:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56e98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.916168:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56e98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760787.981532:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25096192 -> 25100288 00010000:00010000:0.0:1789760787.981552:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56e98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.981565:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56e98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.981573:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56e98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.981580:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760787.981585:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56e98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.981593:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56e98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.981612:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56e98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.981620:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56e98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25096192->25100287] (req 25096192->25100287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.981722:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508471296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 65709us (66086us total) trans 4294973447 rc 0/0 00000100:00100000:0.0:1789760787.981741:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6159 00000100:00100000:0.0:1789760787.981750:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766987063719 time 767020706704 state 0x2 ref 0 00000001:00080000:1.0:1789760787.983328:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973447 is committed 00000100:00100000:0.0:1789760787.985800:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760787.985935:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508471424 00000100:00100000:3.0:1789760787.985945:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508471424 00000100:00100000:3.0:1789760787.986000:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 766987063719 time 767020706704 state 0x0 ref 1 00000100:00100000:3.0:1789760787.986014:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767020967120 time 767020706704 state 0x1 ref 1 00000100:00100000:3.0:1789760787.986044:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767020967120 time 767090754530 state 0x0 ref 1 00000100:00100000:3.0:1789760787.986056:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508471424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760787.986113:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25100288, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:3.0:1789760787.986122:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56e9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.986137:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760787.986142:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.986148:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56e9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.986164:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56e9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760788.025734:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25100288 -> 25104384 00010000:00010000:1.0:1789760788.025752:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56e9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.025764:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56e9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.025774:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56e9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.025781:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760788.025785:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56e9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.025793:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56e9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.025816:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56e9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.025823:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe56e9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25100288->25104383] (req 25100288->25104383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.025923:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508471424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39862us (40126us total) trans 4294973448 rc 0/0 00000100:00100000:1.0:1789760788.025934:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6160 00000100:00100000:1.0:1789760788.025944:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767020967120 time 767090754530 state 0x2 ref 0 00000001:00080000:2.0:1789760788.028961:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973448 is committed 00000100:00100000:2.0:1789760788.029576:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760788.029661:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508471552 00000100:00100000:0.0:1789760788.029669:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508471552 00000100:00100000:0.0:1789760788.029733:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767020967120 time 767090754530 state 0x0 ref 1 00000100:00100000:0.0:1789760788.029748:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767091014946 time 767090754530 state 0x1 ref 1 00000100:00100000:0.0:1789760788.029800:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767091014946 time 767134509766 state 0x0 ref 1 00000100:00100000:0.0:1789760788.029812:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508471552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760788.029858:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25104384, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:0.0:1789760788.029867:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56ea6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.029887:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760788.029892:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.029898:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56ea6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.029911:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56ea6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.061004:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973449 is committed 00002000:00010000:0.0:1789760788.061148:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25104384 -> 25108480 00010000:00010000:0.0:1789760788.061164:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56ea6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.061177:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56ea6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.061186:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56ea6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.061194:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760788.061200:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56ea6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.061209:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56ea6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.061228:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56ea6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.061362:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56ea6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25104384->25108479] (req 25104384->25108479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.061552:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508471552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31736us (31975us total) trans 4294973449 rc 0/0 00000100:00100000:1.0:1789760788.061564:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6161 00000100:00100000:1.0:1789760788.061573:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767091014946 time 767134509766 state 0x2 ref 0 00000100:00100000:3.0:1789760788.066051:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760788.066150:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508471680 00000100:00100000:0.0:1789760788.066161:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508471680 00000100:00100000:0.0:1789760788.066218:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767091014946 time 767134509766 state 0x0 ref 1 00000100:00100000:0.0:1789760788.066243:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767134770182 time 767134509766 state 0x1 ref 1 00000100:00100000:0.0:1789760788.066274:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767134770182 time 767170984412 state 0x0 ref 1 00000100:00100000:0.0:1789760788.066317:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508471680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760788.066365:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25108480, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:0.0:1789760788.066373:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56ead lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.066388:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760788.066394:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.066400:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56ead lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.066415:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56ead lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.107916:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973450 is committed 00002000:00010000:0.0:1789760788.108001:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25108480 -> 25112576 00010000:00010000:0.0:1789760788.108016:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56ead lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.108028:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56ead lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.108037:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56ead lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.108044:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760788.108049:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56ead lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.108059:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56ead lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.108141:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56ead lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.108150:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56ead lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25108480->25112575] (req 25108480->25112575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.108334:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508471680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41951us (42194us total) trans 4294973450 rc 0/0 00000100:00100000:0.0:1789760788.108345:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6162 00000100:00100000:0.0:1789760788.108359:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767134770182 time 767170984412 state 0x2 ref 0 00000100:00100000:0.0:1789760788.112557:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760788.112776:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508471808 00000100:00100000:1.0:1789760788.112786:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508471808 00000100:00100000:1.0:1789760788.112838:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767134770182 time 767170984412 state 0x0 ref 1 00000100:00100000:1.0:1789760788.112853:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767171244828 time 767170984412 state 0x1 ref 1 00000100:00100000:1.0:1789760788.112886:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767171244828 time 767217595926 state 0x0 ref 1 00000100:00100000:1.0:1789760788.112903:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508471808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760788.112953:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25112576, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:1.0:1789760788.112962:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56eb4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.112977:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760788.112981:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.112987:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56eb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.113129:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56eb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.195456:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973451 is committed 00002000:00010000:3.0:1789760788.195832:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25112576 -> 25116672 00010000:00010000:3.0:1789760788.195849:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56eb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.195861:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56eb4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.195870:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56eb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.195876:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760788.195880:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56eb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.195889:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56eb4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.195908:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56eb4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.195914:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe56eb4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25112576->25116671] (req 25112576->25116671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.196010:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508471808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 83106us (83460us total) trans 4294973451 rc 0/0 00000100:00100000:3.0:1789760788.196034:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6163 00000100:00100000:3.0:1789760788.196044:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767171244828 time 767217595926 state 0x2 ref 0 00000100:00100000:2.0:1789760788.199003:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760788.199135:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508471936 00000100:00100000:1.0:1789760788.199146:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508471936 00000100:00100000:1.0:1789760788.199198:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767171244828 time 767217595926 state 0x0 ref 1 00000100:00100000:1.0:1789760788.199212:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767217856342 time 767217595926 state 0x1 ref 1 00000100:00100000:1.0:1789760788.199242:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767217856342 time 767303952079 state 0x0 ref 1 00000100:00100000:1.0:1789760788.199254:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508471936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760788.199334:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25116672, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:1.0:1789760788.199342:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56ebb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.199357:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760788.199362:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.199368:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56ebb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.199381:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56ebb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.224228:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760788.256714:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973452 is committed 00002000:00010000:3.0:1789760788.256809:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25116672 -> 25120768 00010000:00010000:3.0:1789760788.256823:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56ebb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.256837:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56ebb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.256846:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56ebb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.256854:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760788.256859:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56ebb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.256869:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56ebb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.256888:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56ebb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.256897:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe56ebb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25116672->25120767] (req 25116672->25120767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.257037:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508471936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57780us (58037us total) trans 4294973452 rc 0/0 00000100:00100000:3.0:1789760788.257048:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6164 00000100:00100000:3.0:1789760788.257058:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767217856342 time 767303952079 state 0x2 ref 0 00000100:00100000:0.0:1789760788.260731:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760788.260821:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508472064 00000100:00100000:2.0:1789760788.260831:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508472064 00000100:00100000:2.0:1789760788.260875:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767217856342 time 767303952079 state 0x0 ref 1 00000100:00100000:2.0:1789760788.260889:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767304212495 time 767303952079 state 0x1 ref 1 00000100:00100000:2.0:1789760788.260955:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767304212495 time 767365665383 state 0x0 ref 1 00000100:00100000:2.0:1789760788.260966:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508472064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760788.261010:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25120768, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:2.0:1789760788.261027:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56ec2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.261042:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760788.261046:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.261052:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56ec2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.261066:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56ec2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.293311:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973453 is committed 00002000:00010000:2.0:1789760788.293391:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25120768 -> 25124864 00010000:00010000:2.0:1789760788.293406:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56ec2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.293418:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56ec2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.293427:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56ec2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.293435:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760788.293440:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56ec2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.293449:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56ec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.293467:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56ec2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.293475:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe56ec2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25120768->25124863] (req 25120768->25124863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.293563:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048880fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508472064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32595us (32845us total) trans 4294973453 rc 0/0 00000100:00100000:2.0:1789760788.293573:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6165 00000100:00100000:2.0:1789760788.293582:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767304212495 time 767365665383 state 0x2 ref 0 00000100:00100000:0.0:1789760788.297282:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760788.297446:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508472192 00000100:00100000:3.0:1789760788.297456:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508472192 00000100:00100000:3.0:1789760788.297506:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767304212495 time 767365665383 state 0x0 ref 1 00000100:00100000:3.0:1789760788.297522:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767365925799 time 767365665383 state 0x1 ref 1 00000100:00100000:3.0:1789760788.297553:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767365925799 time 767402262993 state 0x0 ref 1 00000100:00100000:3.0:1789760788.297565:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508472192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760788.297613:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25124864, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:3.0:1789760788.297622:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ec9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.297638:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.297642:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.297648:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ec9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.297662:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ec9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760788.328254:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25124864 -> 25128960 00010000:00010000:3.0:1789760788.328273:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ec9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.328316:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ec9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.328325:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ec9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.328333:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760788.328337:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ec9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.328346:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ec9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.328364:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ec9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.328371:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ec9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25124864->25128959] (req 25124864->25128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.328449:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508472192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30882us (31173us total) trans 4294973454 rc 0/0 00000100:00100000:3.0:1789760788.328462:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6166 00000100:00100000:3.0:1789760788.328472:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767365925799 time 767402262993 state 0x2 ref 0 00000001:00080000:3.0:1789760788.328505:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973454 is committed 00000100:00100000:0.0:1789760788.331894:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760788.331967:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508472320 00000100:00100000:3.0:1789760788.331976:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508472320 00000100:00100000:3.0:1789760788.332038:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767365925799 time 767402262993 state 0x0 ref 1 00000100:00100000:3.0:1789760788.332053:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767402523409 time 767402262993 state 0x1 ref 1 00000100:00100000:3.0:1789760788.332089:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767402523409 time 767436799227 state 0x0 ref 1 00000100:00100000:3.0:1789760788.332101:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508472320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760788.332147:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25128960, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:3.0:1789760788.332157:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ed0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.332173:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.332177:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.332183:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ed0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.332198:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ed0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.351658:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7340e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534091136:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760788.351740:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760788.351876:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534091136 00000100:00100000:1.0:1789760788.351884:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534091136 00000100:00100000:1.0:1789760788.351937:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:1.0:1789760788.351947:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534091136:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760788.351985:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760788.352041:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534091136:12345-0@lo:41:osp-pre-0-1.0 Request processed in 95us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760788.352048:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:2.0:1789760788.352318:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7340e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534091136:0@lo:41:osp-pre-0-1.0 00000001:00080000:1.0:1789760788.381651:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973455 is committed 00002000:00010000:3.0:1789760788.381964:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25128960 -> 25133056 00010000:00010000:3.0:1789760788.381981:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ed0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.381993:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ed0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.382002:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ed0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.382010:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760788.382016:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ed0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.382025:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ed0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.382045:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ed0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.382053:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ed0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25128960->25133055] (req 25128960->25133055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.382154:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508472320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50049us (50264us total) trans 4294973455 rc 0/0 00000100:00100000:3.0:1789760788.382165:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6167 00000100:00100000:3.0:1789760788.382174:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767402523409 time 767436799227 state 0x2 ref 0 00000100:00100000:1.0:1789760788.386095:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760788.386231:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508472448 00000100:00100000:2.0:1789760788.386241:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508472448 00000100:00100000:2.0:1789760788.386308:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767402523409 time 767436799227 state 0x0 ref 1 00000100:00100000:2.0:1789760788.386389:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767437059643 time 767436799227 state 0x1 ref 1 00000100:00100000:2.0:1789760788.386417:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767437059643 time 767491127275 state 0x0 ref 1 00000100:00100000:2.0:1789760788.386431:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508472448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760788.386479:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25133056, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:2.0:1789760788.386489:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56ed7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.386505:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760788.386509:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.386516:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56ed7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.386530:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56ed7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.418393:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973456 is committed 00002000:00010000:3.0:1789760788.418499:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25133056 -> 25137152 00010000:00010000:3.0:1789760788.418514:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56ed7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.418527:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56ed7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.418536:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56ed7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.418544:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760788.418548:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56ed7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.418558:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56ed7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.418579:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56ed7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.418587:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe56ed7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25133056->25137151] (req 25133056->25137151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.418806:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508472448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32373us (32716us total) trans 4294973456 rc 0/0 00000100:00100000:3.0:1789760788.418816:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6168 00000100:00100000:3.0:1789760788.418826:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767437059643 time 767491127275 state 0x2 ref 0 00000100:00100000:2.0:1789760788.422822:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760788.423131:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508472576 00000100:00100000:3.0:1789760788.423141:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508472576 00000100:00100000:3.0:1789760788.423193:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767437059643 time 767491127275 state 0x0 ref 1 00000100:00100000:3.0:1789760788.423208:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767491387691 time 767491127275 state 0x1 ref 1 00000100:00100000:3.0:1789760788.423239:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767491387691 time 767527950041 state 0x0 ref 1 00000100:00100000:3.0:1789760788.423253:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508472576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760788.423511:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25137152, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:3.0:1789760788.423523:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ede lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.423537:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.423543:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.423549:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ede lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.423563:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ede lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.453964:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973457 is committed 00002000:00010000:2.0:1789760788.454050:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25137152 -> 25141248 00010000:00010000:2.0:1789760788.454065:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ede lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.454128:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ede lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.454138:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ede lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.454145:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760788.454150:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ede lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.454159:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ede lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.454177:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ede lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.454185:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe56ede lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25137152->25141247] (req 25137152->25141247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.454267:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f4dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508472576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31013us (31450us total) trans 4294973457 rc 0/0 00000100:00100000:2.0:1789760788.454276:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6169 00000100:00100000:2.0:1789760788.454314:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767491387691 time 767527950041 state 0x2 ref 0 00000100:00100000:2.0:1789760788.457248:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760788.457345:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508472704 00000100:00100000:3.0:1789760788.457354:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508472704 00000100:00100000:3.0:1789760788.457406:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767491387691 time 767527950041 state 0x0 ref 1 00000100:00100000:3.0:1789760788.457421:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767528210457 time 767527950041 state 0x1 ref 1 00000100:00100000:3.0:1789760788.457453:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767528210457 time 767562163795 state 0x0 ref 1 00000100:00100000:3.0:1789760788.457466:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508472704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760788.457515:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25141248, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:3.0:1789760788.457524:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ee5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.457539:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.457543:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.457549:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ee5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.457562:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ee5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.492396:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973458 is committed 00002000:00010000:3.0:1789760788.492508:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25141248 -> 25145344 00010000:00010000:3.0:1789760788.492523:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ee5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.492536:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ee5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.492544:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ee5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.492551:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760788.492557:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ee5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.492565:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ee5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.492583:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ee5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.492591:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56ee5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25141248->25145343] (req 25141248->25145343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.492767:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508472704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35297us (35522us total) trans 4294973458 rc 0/0 00000100:00100000:3.0:1789760788.492777:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6170 00000100:00100000:3.0:1789760788.492787:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767528210457 time 767562163795 state 0x2 ref 0 00000100:00100000:3.0:1789760788.495843:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760788.495936:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508472832 00000100:00100000:0.0:1789760788.495946:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508472832 00000100:00100000:0.0:1789760788.495997:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767528210457 time 767562163795 state 0x0 ref 1 00000100:00100000:0.0:1789760788.496011:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767562424211 time 767562163795 state 0x1 ref 1 00000100:00100000:0.0:1789760788.496043:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767562424211 time 767600753876 state 0x0 ref 1 00000100:00100000:0.0:1789760788.496055:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508472832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760788.496105:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25145344, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0xc007 00010000:00010000:0.0:1789760788.496115:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56eec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.496130:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760788.496134:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.496142:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56eec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.496156:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56eec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.526850:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973459 is committed 00002000:00010000:3.0:1789760788.527007:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25145344 -> 25149440 00002000:00010000:3.0:1789760788.527012:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760788 -> 1789760789 00002000:00010000:3.0:1789760788.527015:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760788 -> 1789760789 00010000:00010000:3.0:1789760788.527028:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56eec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.527041:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56eec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.527055:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56eec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.527062:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760788.527067:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56eec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.527085:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56eec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.527104:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56eec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.527112:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56eec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25145344->25149439] (req 25145344->25149439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.527202:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508472832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31143us (31363us total) trans 4294973459 rc 0/0 00000100:00100000:3.0:1789760788.527211:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6171 00000100:00100000:3.0:1789760788.527222:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767562424211 time 767600753876 state 0x2 ref 0 00000100:00100000:2.0:1789760788.530256:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760788.530387:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508472960 00000100:00100000:1.0:1789760788.530396:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508472960 00000100:00100000:1.0:1789760788.530441:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767562424211 time 767600753876 state 0x0 ref 1 00000100:00100000:1.0:1789760788.530456:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767601014292 time 767600753876 state 0x1 ref 1 00000100:00100000:1.0:1789760788.530511:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767601014292 time 767635221723 state 0x0 ref 1 00000100:00100000:1.0:1789760788.530522:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508472960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760788.530567:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25149440, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc007 00010000:00010000:1.0:1789760788.530576:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56ef3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.530592:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760788.530596:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.530601:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56ef3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.530616:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56ef3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.565196:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973460 is committed 00002000:00010000:3.0:1789760788.565313:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25149440 -> 25153536 00010000:00010000:3.0:1789760788.565325:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56ef3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.565338:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56ef3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.565344:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56ef3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.565348:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760788.565351:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56ef3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.565357:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56ef3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.565372:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56ef3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.565377:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe56ef3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25149440->25153535] (req 25149440->25153535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.565487:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508472960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34962us (35236us total) trans 4294973460 rc 0/0 00000100:00100000:3.0:1789760788.565498:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6172 00000100:00100000:3.0:1789760788.565511:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767601014292 time 767635221723 state 0x2 ref 0 00000100:00100000:2.0:1789760788.569759:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760788.570244:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508473088 00000100:00100000:3.0:1789760788.570255:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508473088 00000100:00100000:3.0:1789760788.570350:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767601014292 time 767635221723 state 0x0 ref 1 00000100:00100000:3.0:1789760788.570366:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767635482139 time 767635221723 state 0x1 ref 1 00000100:00100000:3.0:1789760788.570396:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767635482139 time 767675106447 state 0x0 ref 1 00000100:00100000:3.0:1789760788.570411:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508473088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760788.570460:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25153536, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc007 00010000:00010000:3.0:1789760788.570469:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56efa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.570485:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.570490:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.570496:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56efa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.570511:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56efa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.614266:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973461 is committed 00002000:00010000:3.0:1789760788.614423:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25153536 -> 25157632 00010000:00010000:3.0:1789760788.614439:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56efa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.614451:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56efa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.614463:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56efa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.614470:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760788.614475:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56efa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.614483:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56efa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.614504:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56efa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.614511:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56efa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25153536->25157631] (req 25153536->25157631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.614644:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508473088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44225us (44883us total) trans 4294973461 rc 0/0 00000100:00100000:3.0:1789760788.614654:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6173 00000100:00100000:3.0:1789760788.614664:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767635482139 time 767675106447 state 0x2 ref 0 00000100:00100000:0.0:1789760788.622371:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760788.622460:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508473216 00000100:00100000:3.0:1789760788.622469:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508473216 00000100:00100000:3.0:1789760788.622523:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767635482139 time 767675106447 state 0x0 ref 1 00000100:00100000:3.0:1789760788.622538:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767675366863 time 767675106447 state 0x1 ref 1 00000100:00100000:3.0:1789760788.622562:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767675366863 time 767727272924 state 0x0 ref 1 00000100:00100000:3.0:1789760788.622574:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508473216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760788.622619:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25157632, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc007 00010000:00010000:3.0:1789760788.622628:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.622654:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.622658:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.622664:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.622678:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.655027:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973462 is committed 00002000:00010000:3.0:1789760788.655167:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25157632 -> 25161728 00010000:00010000:3.0:1789760788.655182:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.655195:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.655205:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.655212:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760788.655218:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.655227:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.655246:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.655254:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25157632->25161727] (req 25157632->25161727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.655398:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508473216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32820us (33032us total) trans 4294973462 rc 0/0 00000100:00100000:3.0:1789760788.655408:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6174 00000100:00100000:3.0:1789760788.655417:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767675366863 time 767727272924 state 0x2 ref 0 00000100:00100000:2.0:1789760788.660194:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760788.660272:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508473344 00000100:00100000:3.0:1789760788.660282:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508473344 00000100:00100000:3.0:1789760788.660380:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767675366863 time 767727272924 state 0x0 ref 1 00000100:00100000:3.0:1789760788.660396:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767727533340 time 767727272924 state 0x1 ref 1 00000100:00100000:3.0:1789760788.660423:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767727533340 time 767765133707 state 0x0 ref 1 00000100:00100000:3.0:1789760788.660435:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508473344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760788.660497:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25161728, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc007 00010000:00010000:3.0:1789760788.660506:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.660523:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.660527:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.660533:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.660547:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.686312:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973463 is committed 00002000:00010000:2.0:1789760788.686543:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25161728 -> 25165824 00010000:00010000:2.0:1789760788.686559:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.686571:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.686579:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.686586:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760788.686591:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.686599:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.686619:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.686627:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe56f08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25161728->25165823] (req 25161728->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.687007:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508473344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26568us (26818us total) trans 4294973463 rc 0/0 00000100:00100000:2.0:1789760788.687019:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6175 00000100:00100000:2.0:1789760788.687029:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767727533340 time 767765133707 state 0x2 ref 0 00000100:00100000:0.0:1789760788.691191:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760788.691437:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508473472 00000100:00100000:3.0:1789760788.691447:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508473472 00000100:00100000:0.0:1789760788.691574:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767727533340 time 767765133707 state 0x0 ref 1 00000100:00100000:0.0:1789760788.691588:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767765394123 time 767765133707 state 0x1 ref 1 00000100:00100000:0.0:1789760788.691670:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767765394123 time 767796379350 state 0x0 ref 1 00000100:00100000:0.0:1789760788.691682:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488649c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508473472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760788.691740:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25165824, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc007 00010000:00010000:0.0:1789760788.691749:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.691764:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:0.0:1789760788.691768:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.691774:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.691797:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760788.726643:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25165824 -> 25169920 00002000:00010000:1.0:1789760788.726774:0:8108:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:1.0:1789760788.726788:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.726800:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.726808:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.726815:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760788.726820:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.726829:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.726848:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.726856:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25165824->25169919] (req 25165824->25169919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.726953:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488649c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508473472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35268us (35766us total) trans 4294973464 rc 0/0 00000100:00100000:1.0:1789760788.726963:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6176 00000100:00100000:1.0:1789760788.726973:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767765394123 time 767796379350 state 0x2 ref 0 00000001:00080000:3.0:1789760788.727056:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973464 is committed 00000100:00100000:0.0:1789760788.734661:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760788.734940:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508473600 00000100:00100000:3.0:1789760788.734951:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508473600 00000100:00100000:3.0:1789760788.735011:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767765394123 time 767796379350 state 0x0 ref 1 00000100:00100000:3.0:1789760788.735027:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767796639766 time 767796379350 state 0x1 ref 1 00000100:00100000:3.0:1789760788.735223:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767796639766 time 767839932908 state 0x0 ref 1 00000100:00100000:3.0:1789760788.735247:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508473600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760788.735323:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25169920, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:3.0:1789760788.735332:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.735347:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.735352:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.735358:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.735371:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.766977:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973465 is committed 00002000:00010000:3.0:1789760788.767065:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25169920 -> 25174016 00010000:00010000:3.0:1789760788.767264:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.767277:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.767362:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.767371:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760788.767376:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.767385:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.767405:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.767412:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25169920->25174015] (req 25169920->25174015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.767545:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508473600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32305us (32899us total) trans 4294973465 rc 0/0 00000100:00100000:3.0:1789760788.767556:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6177 00000100:00100000:3.0:1789760788.767565:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767796639766 time 767839932908 state 0x2 ref 0 00000100:00100000:0.0:1789760788.773476:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760788.773570:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508473728 00000100:00100000:2.0:1789760788.773580:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508473728 00000100:00100000:2.0:1789760788.773633:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767796639766 time 767839932908 state 0x0 ref 1 00000100:00100000:2.0:1789760788.773647:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767840193324 time 767839932908 state 0x1 ref 1 00000100:00100000:2.0:1789760788.773675:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767840193324 time 767878386512 state 0x0 ref 1 00000100:00100000:2.0:1789760788.773687:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508473728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760788.773744:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25174016, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:2.0:1789760788.773823:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56f1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.773840:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:2.0:1789760788.773846:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.773852:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56f1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.773866:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56f1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.818821:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973466 is committed 00002000:00010000:0.0:1789760788.819021:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25174016 -> 25178112 00010000:00010000:0.0:1789760788.819036:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56f1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819048:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56f1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819056:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56f1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819063:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760788.819067:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56f1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819076:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56f1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819095:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56f1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.819103:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe56f1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25174016->25178111] (req 25174016->25178111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.819195:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508473728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45505us (45724us total) trans 4294973466 rc 0/0 00000100:00100000:0.0:1789760788.819204:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6178 00000100:00100000:0.0:1789760788.819214:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767840193324 time 767878386512 state 0x2 ref 0 00000100:00100000:0.0:1789760788.825865:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760788.825930:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508473856 00000100:00100000:1.0:1789760788.825940:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508473856 00000100:00100000:1.0:1789760788.826004:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767840193324 time 767878386512 state 0x0 ref 1 00000100:00100000:1.0:1789760788.826019:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767878646928 time 767878386512 state 0x1 ref 1 00000100:00100000:1.0:1789760788.826047:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767878646928 time 767930758446 state 0x0 ref 1 00000100:00100000:1.0:1789760788.826058:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048840c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508473856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760788.826228:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25178112, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:1.0:1789760788.826239:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56f24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.826255:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:1.0:1789760788.826260:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.826266:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56f24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.826279:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56f24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760788.861252:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25178112 -> 25182208 00010000:00010000:0.0:1789760788.861271:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56f24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.861311:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56f24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.861322:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56f24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.861331:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760788.861335:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56f24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.861345:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56f24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.861368:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56f24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.861376:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe56f24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25178112->25182207] (req 25178112->25182207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.861463:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048840c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508473856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35401us (35602us total) trans 4294973467 rc 0/0 00000100:00100000:0.0:1789760788.861474:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6179 00000100:00100000:0.0:1789760788.861483:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767878646928 time 767930758446 state 0x2 ref 0 00000001:00080000:3.0:1789760788.861661:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973467 is committed 00000100:00100000:2.0:1789760788.866995:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760788.867549:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508473984 00000100:00100000:3.0:1789760788.867559:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508473984 00000100:00100000:3.0:1789760788.867611:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767878646928 time 767930758446 state 0x0 ref 1 00000100:00100000:3.0:1789760788.867626:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767931018862 time 767930758446 state 0x1 ref 1 00000100:00100000:3.0:1789760788.867661:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767931018862 time 767972371650 state 0x0 ref 1 00000100:00100000:3.0:1789760788.867673:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508473984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760788.867729:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25182208, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:3.0:1789760788.867739:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.867755:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=767s v=(0 0) 00010000:00010000:3.0:1789760788.867760:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.867766:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.867780:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.900671:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973468 is committed 00002000:00010000:3.0:1789760788.900896:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25182208 -> 25186304 00010000:00010000:3.0:1789760788.900911:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.900923:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.900932:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.900938:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760788.900943:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.900951:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.900974:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.900981:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25182208->25186303] (req 25182208->25186303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.901165:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508473984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33489us (34173us total) trans 4294973468 rc 0/0 00000100:00100000:3.0:1789760788.901177:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6180 00000100:00100000:3.0:1789760788.901186:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767931018862 time 767972371650 state 0x2 ref 0 00000100:00100000:1.0:1789760788.905958:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760788.906031:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508474112 00000100:00100000:0.0:1789760788.906040:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508474112 00000100:00100000:0.0:1789760788.906168:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767931018862 time 767972371650 state 0x0 ref 1 00000100:00100000:0.0:1789760788.906183:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767972632066 time 767972371650 state 0x1 ref 1 00000100:00100000:0.0:1789760788.906215:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767972632066 time 768010925754 state 0x0 ref 1 00000100:00100000:0.0:1789760788.906227:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508474112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760788.906275:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25186304, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:0.0:1789760788.906315:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56f32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.906331:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760788.906335:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.906342:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56f32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.906357:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56f32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.945805:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973469 is committed 00002000:00010000:1.0:1789760788.945961:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25186304 -> 25190400 00010000:00010000:1.0:1789760788.945975:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56f32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.945988:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56f32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.945997:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56f32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.946004:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760788.946009:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56f32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.946022:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56f32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.946041:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56f32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.946048:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56f32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25186304->25190399] (req 25186304->25190399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.946155:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508474112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39925us (40203us total) trans 4294973469 rc 0/0 00000100:00100000:1.0:1789760788.946167:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6181 00000100:00100000:1.0:1789760788.946177:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767972632066 time 768010925754 state 0x2 ref 0 00000100:00100000:0.0:1789760788.950251:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760788.951043:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508474240 00000100:00100000:1.0:1789760788.951055:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508474240 00000100:00100000:1.0:1789760788.951221:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 767972632066 time 768010925754 state 0x0 ref 1 00000100:00100000:1.0:1789760788.951236:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768011186170 time 768010925754 state 0x1 ref 1 00000100:00100000:1.0:1789760788.951541:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768011186170 time 768056249140 state 0x0 ref 1 00000100:00100000:1.0:1789760788.951556:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508474240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760788.951604:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25190400, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:1.0:1789760788.951612:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.951640:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760788.951644:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.951651:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.951672:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760788.986041:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25190400 -> 25194496 00010000:00010000:1.0:1789760788.986062:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.986076:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.986086:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.986093:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760788.986098:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.986106:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.986126:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.986134:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25190400->25194495] (req 25190400->25194495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.986226:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508474240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34668us (35980us total) trans 4294973470 rc 0/0 00000100:00100000:1.0:1789760788.986457:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6182 00000100:00100000:1.0:1789760788.986467:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768011186170 time 768056249140 state 0x2 ref 0 00000001:00080000:3.0:1789760788.986555:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973470 is committed 00000100:00100000:0.0:1789760788.996418:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760788.996590:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508474368 00000100:00100000:3.0:1789760788.996601:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508474368 00000100:00100000:3.0:1789760788.996768:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768011186170 time 768056249140 state 0x0 ref 1 00000100:00100000:3.0:1789760788.996842:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768056509556 time 768056249140 state 0x1 ref 1 00000100:00100000:3.0:1789760788.996874:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768056509556 time 768101585405 state 0x0 ref 1 00000100:00100000:3.0:1789760788.996888:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508474368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760788.996935:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25194496, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:3.0:1789760788.996944:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.996959:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760788.996963:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.996970:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.996983:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.042312:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973471 is committed 00002000:00010000:3.0:1789760789.042452:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25194496 -> 25198592 00010000:00010000:3.0:1789760789.042468:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.042480:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.042490:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.042501:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760789.042506:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.042515:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.042534:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.042588:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25194496->25198591] (req 25194496->25198591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.042716:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508474368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45827us (46302us total) trans 4294973471 rc 0/0 00000100:00100000:3.0:1789760789.042851:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6183 00000100:00100000:3.0:1789760789.042861:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768056509556 time 768101585405 state 0x2 ref 0 00000100:00100000:0.0:1789760789.047213:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760789.047455:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508474496 00000100:00100000:1.0:1789760789.047464:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508474496 00000100:00100000:1.0:1789760789.047530:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768056509556 time 768101585405 state 0x0 ref 1 00000100:00100000:1.0:1789760789.047544:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768101845821 time 768101585405 state 0x1 ref 1 00000100:00080000:3.0:1789760789.047616:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760789.047617:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768101845821 time 768152326568 state 0x0 ref 1 00000100:00100000:1.0:1789760789.047629:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c1fec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508474496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760789.047691:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25198592, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:1.0:1789760789.047699:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.047713:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760789.047729:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.047736:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.047750:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.097913:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973472 is committed 00002000:00010000:3.0:1789760789.098039:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25198592 -> 25202688 00010000:00010000:3.0:1789760789.098055:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.098076:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.098086:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.098093:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760789.098198:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.098209:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.098229:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.098237:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe56f47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25198592->25202687] (req 25198592->25202687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.098576:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c1fec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508474496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50942us (51366us total) trans 4294973472 rc 0/0 00000100:00100000:3.0:1789760789.098589:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6184 00000100:00100000:3.0:1789760789.098599:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768101845821 time 768152326568 state 0x2 ref 0 00000100:00100000:0.0:1789760789.102926:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760789.103915:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508474624 00000100:00100000:1.0:1789760789.103929:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508474624 00000100:00100000:1.0:1789760789.103984:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768101845821 time 768152326568 state 0x0 ref 1 00000100:00100000:1.0:1789760789.103999:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768152586984 time 768152326568 state 0x1 ref 1 00000100:00100000:1.0:1789760789.104036:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768152586984 time 768208746239 state 0x0 ref 1 00000100:00100000:1.0:1789760789.104048:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508474624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760789.104209:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25202688, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:1.0:1789760789.104220:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56f4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.104237:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760789.104241:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.104248:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56f4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.104262:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56f4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760789.148755:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25202688 -> 25206784 00010000:00010000:0.0:1789760789.148775:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56f4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.148798:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56f4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.148808:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56f4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.148816:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760789.148820:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56f4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.148829:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56f4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.148848:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56f4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.148856:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe56f4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25202688->25206783] (req 25202688->25206783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.148874:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973473 is committed 00000100:00100000:0.0:1789760789.148946:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508474624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44895us (46027us total) trans 4294973473 rc 0/0 00000100:00100000:0.0:1789760789.148960:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6185 00000100:00100000:0.0:1789760789.148969:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768152586984 time 768208746239 state 0x2 ref 0 00000100:00100000:1.0:1789760789.156955:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760789.157578:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508474752 00000100:00100000:3.0:1789760789.157588:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508474752 00000100:00100000:3.0:1789760789.157776:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768152586984 time 768208746239 state 0x0 ref 1 00000100:00100000:3.0:1789760789.157791:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768209006655 time 768208746239 state 0x1 ref 1 00000100:00100000:3.0:1789760789.157860:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768209006655 time 768262569798 state 0x0 ref 1 00000100:00100000:3.0:1789760789.157873:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508474752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760789.157919:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25206784, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:3.0:1789760789.157928:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.157944:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760789.157948:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.157955:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.157968:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760789.218151:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25206784 -> 25210880 00010000:00010000:3.0:1789760789.218170:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.218183:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.218191:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.218198:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760789.218203:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.218211:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.218229:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.218238:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25206784->25210879] (req 25206784->25210879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.218422:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508474752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60547us (61471us total) trans 4294973474 rc 0/0 00000100:00100000:3.0:1789760789.218436:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6186 00000100:00100000:3.0:1789760789.218455:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768209006655 time 768262569798 state 0x2 ref 0 00000001:00080000:3.0:1789760789.218492:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973474 is committed 00000100:00100000:1.0:1789760789.223872:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760789.225457:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508474880 00000100:00100000:3.0:1789760789.225471:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508474880 00000100:00100000:3.0:1789760789.225528:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768209006655 time 768262569798 state 0x0 ref 1 00000100:00100000:3.0:1789760789.225550:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768262830214 time 768262569798 state 0x1 ref 1 00000100:00100000:3.0:1789760789.225641:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768262830214 time 768330350840 state 0x0 ref 1 00000100:00100000:3.0:1789760789.225655:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508474880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760789.225705:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25210880, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:3.0:1789760789.225714:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.225738:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760789.225743:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.225750:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.225763:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.246700:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760789.259199:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973475 is committed 00002000:00010000:3.0:1789760789.259503:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25210880 -> 25214976 00010000:00010000:3.0:1789760789.259518:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.259529:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.259536:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.259543:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760789.259547:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.259556:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.259577:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.259585:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25210880->25214975] (req 25210880->25214975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.259901:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508474880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34243us (36034us total) trans 4294973475 rc 0/0 00000100:00100000:3.0:1789760789.259912:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6187 00000100:00100000:3.0:1789760789.259921:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768262830214 time 768330350840 state 0x2 ref 0 00000100:00100000:3.0:1789760789.265509:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760789.265805:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508475008 00000100:00100000:0.0:1789760789.265815:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508475008 00000100:00100000:0.0:1789760789.265868:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768262830214 time 768330350840 state 0x0 ref 1 00000100:00100000:0.0:1789760789.265883:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768330611256 time 768330350840 state 0x1 ref 1 00000100:00100000:0.0:1789760789.265919:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768330611256 time 768370629611 state 0x0 ref 1 00000100:00100000:0.0:1789760789.265932:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508475008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760789.265979:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25214976, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:0.0:1789760789.265987:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.266004:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760789.266009:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.266014:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.266029:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760789.309604:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25214976 -> 25219072 00010000:00010000:1.0:1789760789.309625:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.309638:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.309705:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.309713:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760789.309726:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.309736:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.309758:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.309765:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe56f63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25214976->25219071] (req 25214976->25219071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.309903:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508475008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43967us (44398us total) trans 4294973476 rc 0/0 00000100:00100000:1.0:1789760789.309915:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6188 00000100:00100000:1.0:1789760789.309926:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768330611256 time 768370629611 state 0x2 ref 0 00000001:00080000:1.0:1789760789.309967:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973476 is committed 00000100:00100000:1.0:1789760789.314419:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760789.314504:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508475136 00000100:00100000:3.0:1789760789.314515:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508475136 00000100:00100000:3.0:1789760789.314630:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768330611256 time 768370629611 state 0x0 ref 1 00000100:00100000:3.0:1789760789.314709:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768370890027 time 768370629611 state 0x1 ref 1 00000100:00100000:3.0:1789760789.314782:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768370890027 time 768419492467 state 0x0 ref 1 00000100:00100000:3.0:1789760789.314795:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508475136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760789.314844:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25219072, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:3.0:1789760789.314853:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.314868:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760789.314872:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.314879:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.314892:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.368691:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973477 is committed 00002000:00010000:0.0:1789760789.368804:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25219072 -> 25223168 00010000:00010000:0.0:1789760789.368819:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.368832:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.368841:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.368849:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760789.368854:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.368864:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.368884:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.368892:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe56f6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25219072->25223167] (req 25219072->25223167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.368983:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508475136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54184us (54568us total) trans 4294973477 rc 0/0 00000100:00100000:0.0:1789760789.368992:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6189 00000100:00100000:0.0:1789760789.369002:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768370890027 time 768419492467 state 0x2 ref 0 00000100:00100000:0.0:1789760789.373817:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760789.373965:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508475264 00000100:00100000:3.0:1789760789.373976:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508475264 00000100:00100000:3.0:1789760789.374043:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768370890027 time 768419492467 state 0x0 ref 1 00000100:00100000:3.0:1789760789.374060:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768419752883 time 768419492467 state 0x1 ref 1 00000100:00100000:3.0:1789760789.374101:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768419752883 time 768478810412 state 0x0 ref 1 00000100:00100000:3.0:1789760789.374114:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508475264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760789.374170:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25223168, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:3.0:1789760789.374179:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56f71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.374196:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760789.374201:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.374207:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56f71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.374221:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56f71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.464023:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973478 is committed 00002000:00010000:2.0:1789760789.464796:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25223168 -> 25227264 00010000:00010000:2.0:1789760789.464814:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56f71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.464826:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56f71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.464836:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56f71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.464843:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760789.464847:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56f71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.464856:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56f71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.464878:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56f71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.464890:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe56f71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25223168->25227263] (req 25223168->25227263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.464992:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508475264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 90875us (91182us total) trans 4294973478 rc 0/0 00000100:00100000:2.0:1789760789.465001:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6190 00000100:00100000:2.0:1789760789.465010:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768419752883 time 768478810412 state 0x2 ref 0 00000100:00100000:1.0:1789760789.472471:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760789.472750:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508475392 00000100:00100000:2.0:1789760789.472760:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508475392 00000100:00100000:2.0:1789760789.472810:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768419752883 time 768478810412 state 0x0 ref 1 00000100:00100000:2.0:1789760789.472825:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768479070828 time 768478810412 state 0x1 ref 1 00000100:00100000:2.0:1789760789.472847:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768479070828 time 768577557078 state 0x0 ref 1 00000100:00100000:2.0:1789760789.472858:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508475392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760789.472907:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25227264, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0xc807 00010000:00010000:2.0:1789760789.472915:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.472931:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760789.472936:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.472942:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.472961:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760789.530543:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25227264 -> 25231360 00002000:00010000:2.0:1789760789.530551:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760789 -> 1789760790 00002000:00010000:2.0:1789760789.530554:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760789 -> 1789760790 00010000:00010000:2.0:1789760789.530565:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.530577:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.530586:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.530593:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760789.530598:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.530607:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.530637:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.530650:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25227264->25231359] (req 25227264->25231359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.530754:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508475392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57892us (58288us total) trans 4294973479 rc 0/0 00000100:00100000:2.0:1789760789.530764:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6191 00000100:00100000:2.0:1789760789.530774:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768479070828 time 768577557078 state 0x2 ref 0 00000001:00080000:0.0:1789760789.531042:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973479 is committed 00000100:00100000:0.0:1789760789.534424:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760789.534631:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508475520 00000100:00100000:3.0:1789760789.534642:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508475520 00000100:00100000:3.0:1789760789.534694:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768479070828 time 768577557078 state 0x0 ref 1 00000100:00100000:3.0:1789760789.534708:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768577817494 time 768577557078 state 0x1 ref 1 00000100:00100000:3.0:1789760789.534749:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768577817494 time 768639459313 state 0x0 ref 1 00000100:00100000:3.0:1789760789.534762:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508475520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760789.534809:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25231360, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:3.0:1789760789.534819:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56f7f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.534835:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:3.0:1789760789.534839:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.534845:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56f7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.534859:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56f7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.578036:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973480 is committed 00002000:00010000:3.0:1789760789.578109:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25231360 -> 25235456 00010000:00010000:3.0:1789760789.578566:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56f7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.578582:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56f7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.578592:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56f7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.578600:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760789.578605:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56f7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.578614:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56f7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.578636:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56f7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.578644:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe56f7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25231360->25235455] (req 25231360->25235455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.578746:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508475520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43981us (44328us total) trans 4294973480 rc 0/0 00000100:00100000:3.0:1789760789.578757:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6192 00000100:00100000:3.0:1789760789.578767:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768577817494 time 768639459313 state 0x2 ref 0 00000100:00100000:3.0:1789760789.584669:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760789.586503:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508475648 00000100:00100000:2.0:1789760789.586519:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508475648 00000100:00100000:2.0:1789760789.586579:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768577817494 time 768639459313 state 0x0 ref 1 00000100:00100000:2.0:1789760789.586595:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768639719729 time 768639459313 state 0x1 ref 1 00000100:00100000:0.0:1789760789.587574:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768639719729 time 768692276393 state 0x0 ref 1 00000100:00100000:0.0:1789760789.587595:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508475648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760789.587647:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25235456, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:0.0:1789760789.587656:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.587672:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760789.587676:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.587683:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.587697:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.635121:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973481 is committed 00002000:00010000:0.0:1789760789.635350:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25235456 -> 25239552 00010000:00010000:0.0:1789760789.635365:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.635378:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.635387:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.635400:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760789.635405:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.635414:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.635433:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.635440:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25235456->25239551] (req 25235456->25239551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.635520:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508475648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47928us (50856us total) trans 4294973481 rc 0/0 00000100:00100000:0.0:1789760789.635531:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6193 00000100:00100000:0.0:1789760789.635540:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768639719729 time 768692276393 state 0x2 ref 0 00000100:00100000:3.0:1789760789.641023:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760789.641179:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508475776 00000100:00100000:2.0:1789760789.641199:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508475776 00000100:00100000:2.0:1789760789.641255:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768639719729 time 768692276393 state 0x0 ref 1 00000100:00100000:2.0:1789760789.641270:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768692536809 time 768692276393 state 0x1 ref 1 00000100:00100000:2.0:1789760789.641432:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768692536809 time 768746140760 state 0x0 ref 1 00000100:00100000:2.0:1789760789.641446:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508475776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760789.641493:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25239552, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:2.0:1789760789.641502:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.641517:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:2.0:1789760789.641521:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.641527:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.641541:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.684156:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973482 is committed 00002000:00010000:1.0:1789760789.684260:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25239552 -> 25243648 00010000:00010000:1.0:1789760789.684275:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.684319:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.684330:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.684337:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760789.684342:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.684352:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.684377:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.684386:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe56f8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25239552->25243647] (req 25239552->25243647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.684477:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508475776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43029us (43460us total) trans 4294973482 rc 0/0 00000100:00100000:1.0:1789760789.684488:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6194 00000100:00100000:1.0:1789760789.684497:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768692536809 time 768746140760 state 0x2 ref 0 00000100:00100000:3.0:1789760789.687768:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760789.687870:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508475904 00000100:00100000:0.0:1789760789.687879:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508475904 00000100:00100000:0.0:1789760789.687935:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768692536809 time 768746140760 state 0x0 ref 1 00000100:00100000:0.0:1789760789.687949:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768746401176 time 768746140760 state 0x1 ref 1 00000100:00100000:0.0:1789760789.687974:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768746401176 time 768792684557 state 0x0 ref 1 00000100:00100000:0.0:1789760789.687989:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508475904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760789.688037:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25243648, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:0.0:1789760789.688045:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.688071:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760789.688076:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.688082:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.688094:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760789.723531:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25243648 -> 25247744 00010000:00010000:0.0:1789760789.723552:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.723566:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.723577:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.723585:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760789.723590:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.723600:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.723621:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.723629:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56f94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25243648->25247743] (req 25243648->25247743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.723725:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508475904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35722us (35952us total) trans 4294973483 rc 0/0 00000100:00100000:0.0:1789760789.723735:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6195 00000100:00100000:0.0:1789760789.723745:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768746401176 time 768792684557 state 0x2 ref 0 00000001:00080000:3.0:1789760789.723850:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973483 is committed 00000100:00100000:3.0:1789760789.726707:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760789.726796:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508476032 00000100:00100000:0.0:1789760789.726806:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508476032 00000100:00100000:0.0:1789760789.726856:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768746401176 time 768792684557 state 0x0 ref 1 00000100:00100000:0.0:1789760789.726871:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768792944973 time 768792684557 state 0x1 ref 1 00000100:00100000:0.0:1789760789.726901:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768792944973 time 768831610969 state 0x0 ref 1 00000100:00100000:0.0:1789760789.726912:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508476032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760789.726959:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25247744, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:0.0:1789760789.726968:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56f9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.726984:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760789.726989:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.726995:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56f9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.727009:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56f9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.762104:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973484 is committed 00002000:00010000:2.0:1789760789.762369:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25247744 -> 25251840 00010000:00010000:2.0:1789760789.762385:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56f9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.762397:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56f9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.762406:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56f9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.762413:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760789.762418:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56f9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.762427:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56f9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.762445:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56f9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.762453:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe56f9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25247744->25251839] (req 25247744->25251839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.762540:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508476032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35625us (35838us total) trans 4294973484 rc 0/0 00000100:00100000:2.0:1789760789.762550:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6196 00000100:00100000:2.0:1789760789.762559:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768792944973 time 768831610969 state 0x2 ref 0 00000100:00100000:3.0:1789760789.774461:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760789.774553:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508476160 00000100:00100000:1.0:1789760789.774569:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508476160 00000100:00100000:1.0:1789760789.774630:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768792944973 time 768831610969 state 0x0 ref 1 00000100:00100000:1.0:1789760789.774645:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768831871385 time 768831610969 state 0x1 ref 1 00000100:00100000:1.0:1789760789.774678:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768831871385 time 768879388868 state 0x0 ref 1 00000100:00100000:1.0:1789760789.774691:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508476160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760789.774751:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25251840, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:1.0:1789760789.774761:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fa2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.774778:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:1.0:1789760789.774782:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.774789:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.774813:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760789.837266:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25251840 -> 25255936 00000001:00080000:0.0:1789760789.837391:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973485 is committed 00010000:00010000:1.0:1789760789.837513:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.837530:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fa2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.837540:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fa2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.837549:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760789.837555:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fa2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.837565:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fa2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.837591:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fa2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.837599:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fa2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25251840->25255935] (req 25251840->25255935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.837859:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508476160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 63162us (63405us total) trans 4294973485 rc 0/0 00000100:00100000:1.0:1789760789.837871:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6197 00000100:00100000:1.0:1789760789.837885:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768831871385 time 768879388868 state 0x2 ref 0 00000100:00100000:3.0:1789760789.841811:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760789.841984:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508476288 00000100:00100000:0.0:1789760789.841997:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508476288 00000100:00100000:0.0:1789760789.842076:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768831871385 time 768879388868 state 0x0 ref 1 00000100:00100000:0.0:1789760789.842095:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768879649284 time 768879388868 state 0x1 ref 1 00000100:00100000:0.0:1789760789.842124:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768879649284 time 768946834497 state 0x0 ref 1 00000100:00100000:0.0:1789760789.842136:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508476288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760789.842198:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25255936, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:0.0:1789760789.842209:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56fa9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.842228:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=768s v=(0 0) 00010000:00010000:0.0:1789760789.842233:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.842240:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56fa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.842257:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56fa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760789.892276:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25255936 -> 25260032 00010000:00010000:1.0:1789760789.892340:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56fa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.892357:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56fa9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.892366:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56fa9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.892374:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760789.892379:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56fa9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.892389:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56fa9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.892413:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56fa9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.892426:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe56fa9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25255936->25260031] (req 25255936->25260031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.892545:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508476288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50402us (50743us total) trans 4294973486 rc 0/0 00000100:00100000:1.0:1789760789.892557:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6198 00000100:00100000:1.0:1789760789.892570:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768879649284 time 768946834497 state 0x2 ref 0 00000001:00080000:2.0:1789760789.892841:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973486 is committed 00000100:00100000:3.0:1789760789.897490:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760789.897818:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508476416 00000100:00100000:2.0:1789760789.897831:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508476416 00000100:00100000:2.0:1789760789.897901:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768879649284 time 768946834497 state 0x0 ref 1 00000100:00100000:2.0:1789760789.897921:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768947094913 time 768946834497 state 0x1 ref 1 00000100:00100000:2.0:1789760789.897955:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768947094913 time 769002665064 state 0x0 ref 1 00000100:00100000:2.0:1789760789.897976:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508476416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760789.898040:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25260032, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:2.0:1789760789.898051:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56fb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.898072:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760789.898077:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.898084:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56fb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.898105:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56fb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.931136:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973487 is committed 00002000:00010000:0.0:1789760789.931368:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25260032 -> 25264128 00010000:00010000:0.0:1789760789.931389:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56fb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.931404:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56fb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.931414:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56fb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.931423:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760789.931429:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56fb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.931439:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56fb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.931465:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56fb0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.931474:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe56fb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25260032->25264127] (req 25260032->25264127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.931599:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508476416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33624us (34116us total) trans 4294973487 rc 0/0 00000100:00100000:0.0:1789760789.931610:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6199 00000100:00100000:0.0:1789760789.931623:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768947094913 time 769002665064 state 0x2 ref 0 00000100:00100000:0.0:1789760789.939610:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760789.942520:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508476544 00000100:00100000:1.0:1789760789.942548:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508476544 00000100:00100000:1.0:1789760789.942805:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 768947094913 time 769002665064 state 0x0 ref 1 00000100:00100000:1.0:1789760789.942830:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769002925480 time 769002665064 state 0x1 ref 1 00000100:00100000:1.0:1789760789.942878:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769002925480 time 769047587538 state 0x0 ref 1 00000100:00100000:1.0:1789760789.942898:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508476544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760789.942977:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25264128, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:1.0:1789760789.942989:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fb7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.943011:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760789.943016:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.943024:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.943047:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.033642:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973488 is committed 00002000:00010000:1.0:1789760790.033757:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25264128 -> 25268224 00010000:00010000:1.0:1789760790.033777:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.033792:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fb7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.033802:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.033809:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760790.033814:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.033823:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fb7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.033849:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fb7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.033858:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fb7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25264128->25268223] (req 25264128->25268223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.034101:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508476544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 91202us (94501us total) trans 4294973488 rc 0/0 00000100:00100000:1.0:1789760790.034112:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6200 00000100:00100000:1.0:1789760790.034124:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769002925480 time 769047587538 state 0x2 ref 0 00000100:00100000:3.0:1789760790.038583:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760790.038906:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508476672 00000100:00100000:1.0:1789760790.038919:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508476672 00000100:00100000:1.0:1789760790.038991:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769002925480 time 769047587538 state 0x0 ref 1 00000100:00100000:1.0:1789760790.039009:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769047847954 time 769047587538 state 0x1 ref 1 00000100:00100000:1.0:1789760790.039086:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769047847954 time 769143796579 state 0x0 ref 1 00000100:00100000:1.0:1789760790.039100:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508476672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760790.039166:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25268224, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:1.0:1789760790.039176:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56fbe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.039195:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760790.039201:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.039208:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56fbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.039224:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56fbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.099525:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973489 is committed 00002000:00010000:1.0:1789760790.099845:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25268224 -> 25272320 00010000:00010000:1.0:1789760790.099865:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56fbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.099881:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56fbe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.099891:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56fbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.099899:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760790.099905:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56fbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.099917:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56fbe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.099942:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56fbe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.099951:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe56fbe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25268224->25272319] (req 25268224->25272319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.100202:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508476672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61097us (61626us total) trans 4294973489 rc 0/0 00000100:00100000:1.0:1789760790.100218:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6201 00000100:00100000:1.0:1789760790.100231:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769047847954 time 769143796579 state 0x2 ref 0 00000100:00100000:3.0:1789760790.103425:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760790.103623:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508476800 00000100:00100000:0.0:1789760790.103642:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508476800 00000100:00100000:0.0:1789760790.103817:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769047847954 time 769143796579 state 0x0 ref 1 00000100:00100000:0.0:1789760790.103835:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769144056995 time 769143796579 state 0x1 ref 1 00000100:00100000:0.0:1789760790.103908:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769144056995 time 769208617360 state 0x0 ref 1 00000100:00100000:0.0:1789760790.103922:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508476800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760790.103984:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25272320, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:0.0:1789760790.103995:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56fc5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.104016:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.104021:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.104027:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56fc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.104043:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56fc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.151439:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973490 is committed 00002000:00010000:2.0:1789760790.151525:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25272320 -> 25276416 00010000:00010000:2.0:1789760790.151547:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56fc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.151563:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56fc5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.151572:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56fc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.151581:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760790.151587:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56fc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.151602:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56fc5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.151627:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56fc5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.151636:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe56fc5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25272320->25276415] (req 25272320->25276415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.151850:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508476800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47922us (48429us total) trans 4294973490 rc 0/0 00000100:00100000:2.0:1789760790.151863:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6202 00000100:00100000:2.0:1789760790.151875:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769144056995 time 769208617360 state 0x2 ref 0 00000100:00100000:0.0:1789760790.155537:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760790.155767:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508476928 00000100:00100000:1.0:1789760790.155781:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508476928 00000100:00100000:1.0:1789760790.155855:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769144056995 time 769208617360 state 0x0 ref 1 00000100:00100000:1.0:1789760790.155873:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769208877776 time 769208617360 state 0x1 ref 1 00000100:00100000:3.0:1789760790.156027:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769208877776 time 769260734742 state 0x0 ref 1 00000100:00100000:3.0:1789760790.156046:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508476928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760790.156183:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25276416, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:3.0:1789760790.156194:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56fcc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.156212:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:3.0:1789760790.156216:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.156222:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56fcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.156235:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56fcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760790.190975:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25276416 -> 25280512 00010000:00010000:0.0:1789760790.191004:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56fcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.191021:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56fcc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.191032:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56fcc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.191040:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760790.191046:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56fcc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.191068:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56fcc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.191095:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56fcc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.191114:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe56fcc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25276416->25280511] (req 25276416->25280511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.191268:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508476928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35217us (35734us total) trans 4294973491 rc 0/0 00000100:00100000:0.0:1789760790.191283:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6203 00000100:00100000:0.0:1789760790.191345:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769208877776 time 769260734742 state 0x2 ref 0 00000001:00080000:3.0:1789760790.192281:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973491 is committed 00000100:00100000:3.0:1789760790.200594:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760790.200737:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508477056 00000100:00100000:1.0:1789760790.200749:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508477056 00000100:00100000:1.0:1789760790.200820:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769208877776 time 769260734742 state 0x0 ref 1 00000100:00100000:1.0:1789760790.200839:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769260995158 time 769260734742 state 0x1 ref 1 00000100:00100000:1.0:1789760790.200873:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769260995158 time 769305583465 state 0x0 ref 1 00000100:00100000:1.0:1789760790.200886:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508477056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760790.200948:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25280512, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:1.0:1789760790.200959:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fd3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.200977:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760790.200982:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.200990:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.201024:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.233802:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973492 is committed 00002000:00010000:1.0:1789760790.233911:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25280512 -> 25284608 00010000:00010000:1.0:1789760790.233931:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.233946:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fd3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.233956:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.233964:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760790.233970:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.233981:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fd3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.234006:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fd3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.234015:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56fd3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25280512->25284607] (req 25280512->25284607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.234467:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508477056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33573us (33877us total) trans 4294973492 rc 0/0 00000100:00100000:1.0:1789760790.234483:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6204 00000100:00100000:1.0:1789760790.234496:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769260995158 time 769305583465 state 0x2 ref 0 00000100:00100000:1.0:1789760790.242917:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760790.243743:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508477184 00000100:00100000:2.0:1789760790.243758:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508477184 00000100:00100000:2.0:1789760790.243842:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769260995158 time 769305583465 state 0x0 ref 1 00000100:00100000:2.0:1789760790.243861:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769305843881 time 769305583465 state 0x1 ref 1 00000100:00100000:2.0:1789760790.243903:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769305843881 time 769348612881 state 0x0 ref 1 00000100:00100000:2.0:1789760790.243918:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508477184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760790.243984:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25284608, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:2.0:1789760790.243995:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56fda lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.244015:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760790.244020:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.244027:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56fda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.244043:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56fda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.272420:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760790.309969:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973493 is committed 00002000:00010000:0.0:1789760790.310074:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25284608 -> 25288704 00010000:00010000:0.0:1789760790.310094:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56fda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.310110:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56fda lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.310121:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56fda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.310129:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760790.310134:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56fda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.310144:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56fda lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.310170:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56fda lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.310178:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe56fda lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25284608->25288703] (req 25284608->25288703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.310435:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508477184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 66514us (67528us total) trans 4294973493 rc 0/0 00000100:00100000:0.0:1789760790.310458:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6205 00000100:00100000:0.0:1789760790.310471:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769305843881 time 769348612881 state 0x2 ref 0 00000100:00100000:2.0:1789760790.312780:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760790.312871:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508477312 00000100:00100000:0.0:1789760790.312884:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508477312 00000100:00100000:0.0:1789760790.313153:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769305843881 time 769348612881 state 0x0 ref 1 00000100:00100000:0.0:1789760790.313172:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769348873297 time 769348612881 state 0x1 ref 1 00000100:00100000:0.0:1789760790.313209:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769348873297 time 769417918968 state 0x0 ref 1 00000100:00100000:0.0:1789760790.313223:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508477312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760790.313381:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25288704, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:0.0:1789760790.313391:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56fe1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.313410:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.313416:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.313425:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56fe1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.313452:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56fe1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.361496:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973494 is committed 00002000:00010000:1.0:1789760790.361812:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25288704 -> 25292800 00010000:00010000:1.0:1789760790.361843:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56fe1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.361858:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56fe1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.361868:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56fe1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.361876:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760790.361882:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56fe1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.361893:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56fe1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.361919:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56fe1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.361929:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe56fe1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25288704->25292799] (req 25288704->25292799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.362054:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508477312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48825us (49278us total) trans 4294973494 rc 0/0 00000100:00100000:1.0:1789760790.362067:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6206 00000100:00100000:1.0:1789760790.362081:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769348873297 time 769417918968 state 0x2 ref 0 00000100:00100000:3.0:1789760790.366003:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760790.366172:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508477440 00000100:00100000:1.0:1789760790.366184:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508477440 00000100:00100000:1.0:1789760790.366255:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769348873297 time 769417918968 state 0x0 ref 1 00000100:00100000:1.0:1789760790.366273:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769418179384 time 769417918968 state 0x1 ref 1 00000100:00100000:2.0:1789760790.366863:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769418179384 time 769471569194 state 0x0 ref 1 00000100:00100000:2.0:1789760790.366884:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508477440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760790.366945:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25292800, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:2.0:1789760790.366955:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56fe8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.366984:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760790.366989:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.366997:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56fe8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.367015:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56fe8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.398455:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760790.398487:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888133100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534091264:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760790.398660:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760790.398759:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534091264 00000100:00100000:1.0:1789760790.398768:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534091264 00000100:00100000:1.0:1789760790.398832:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:1.0:1789760790.398844:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534091264:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760790.398900:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760790.398963:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534091264:12345-0@lo:41:osp-pre-1-0.0 Request processed in 119us (301us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760790.398972:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:1.0:1789760790.399035:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888133100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534091264:0@lo:41:osp-pre-1-0.0 00000001:00080000:3.0:1789760790.418857:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973495 is committed 00002000:00010000:1.0:1789760790.419002:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25292800 -> 25296896 00010000:00010000:1.0:1789760790.419022:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56fe8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.419039:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56fe8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.419049:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56fe8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.419057:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760790.419062:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56fe8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.419072:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56fe8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.419100:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56fe8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.419109:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe56fe8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25292800->25296895] (req 25292800->25296895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.419363:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508477440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52476us (53365us total) trans 4294973495 rc 0/0 00000100:00100000:1.0:1789760790.419375:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6207 00000100:00100000:1.0:1789760790.419782:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769418179384 time 769471569194 state 0x2 ref 0 00000100:00100000:1.0:1789760790.424698:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760790.424787:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508477568 00000100:00100000:0.0:1789760790.424800:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508477568 00000100:00100000:0.0:1789760790.424876:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769418179384 time 769471569194 state 0x0 ref 1 00000100:00100000:0.0:1789760790.424897:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769471829610 time 769471569194 state 0x1 ref 1 00000100:00100000:0.0:1789760790.424933:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769471829610 time 769529643532 state 0x0 ref 1 00000100:00100000:0.0:1789760790.424947:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888133100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508477568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760790.425020:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25296896, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:0.0:1789760790.425033:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56fef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.425070:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.425077:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.425086:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56fef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.425106:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56fef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.486716:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973496 is committed 00002000:00010000:1.0:1789760790.486906:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25296896 -> 25300992 00010000:00010000:1.0:1789760790.486929:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56fef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.486945:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56fef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.486955:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56fef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.486962:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760790.486968:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56fef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.486979:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56fef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.487006:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56fef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.487015:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe56fef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25296896->25300991] (req 25296896->25300991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.487173:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888133100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508477568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 62219us (62480us total) trans 4294973496 rc 0/0 00000100:00100000:1.0:1789760790.487190:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6208 00000100:00100000:1.0:1789760790.487203:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769471829610 time 769529643532 state 0x2 ref 0 00000100:00100000:3.0:1789760790.491043:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760790.491189:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508477696 00000100:00100000:1.0:1789760790.491204:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508477696 00000100:00100000:1.0:1789760790.491337:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769471829610 time 769529643532 state 0x0 ref 1 00000100:00100000:1.0:1789760790.491356:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769529903948 time 769529643532 state 0x1 ref 1 00000100:00100000:1.0:1789760790.491386:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769529903948 time 769596096466 state 0x0 ref 1 00000100:00100000:1.0:1789760790.491401:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508477696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760790.491468:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25300992, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0xc807 00010000:00010000:1.0:1789760790.491478:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56ff6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.491499:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760790.491504:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.491512:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56ff6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.491529:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56ff6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760790.539512:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25300992 -> 25305088 00002000:00010000:1.0:1789760790.539523:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760790 -> 1789760791 00002000:00010000:1.0:1789760790.539527:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760790 -> 1789760791 00010000:00010000:1.0:1789760790.539543:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56ff6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.539560:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56ff6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.539570:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56ff6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.539577:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760790.539583:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56ff6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.539594:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56ff6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.539628:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56ff6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.539636:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe56ff6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25300992->25305087] (req 25300992->25305087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.539765:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508477696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48360us (48731us total) trans 4294973497 rc 0/0 00000100:00100000:1.0:1789760790.539776:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6209 00000100:00100000:1.0:1789760790.539787:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769529903948 time 769596096466 state 0x2 ref 0 00000001:00080000:0.0:1789760790.539937:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973497 is committed 00000100:00100000:3.0:1789760790.542526:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760790.542641:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508477824 00000100:00100000:1.0:1789760790.542653:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508477824 00000100:00100000:1.0:1789760790.542737:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769529903948 time 769596096466 state 0x0 ref 1 00000100:00100000:1.0:1789760790.542756:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769596356882 time 769596096466 state 0x1 ref 1 00000100:00100000:1.0:1789760790.542788:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769596356882 time 769647498348 state 0x0 ref 1 00000100:00100000:1.0:1789760790.542802:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508477824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760790.542867:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25305088, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:1.0:1789760790.542877:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56ffd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.542898:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760790.542904:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.542911:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56ffd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.542928:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56ffd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.572023:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973498 is committed 00002000:00010000:2.0:1789760790.572244:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25305088 -> 25309184 00010000:00010000:2.0:1789760790.572264:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56ffd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.572279:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56ffd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.572313:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56ffd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.572323:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760790.572329:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56ffd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.572338:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56ffd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.572363:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56ffd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.572373:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe56ffd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25305088->25309183] (req 25305088->25309183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.572499:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508477824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29692us (29978us total) trans 4294973498 rc 0/0 00000100:00100000:2.0:1789760790.572511:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6210 00000100:00100000:2.0:1789760790.572523:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769596356882 time 769647498348 state 0x2 ref 0 00000100:00100000:3.0:1789760790.576168:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760790.576534:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508477952 00000100:00100000:0.0:1789760790.576549:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508477952 00000100:00100000:0.0:1789760790.576683:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769596356882 time 769647498348 state 0x0 ref 1 00000100:00100000:0.0:1789760790.576719:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769647758764 time 769647498348 state 0x1 ref 1 00000100:00100000:0.0:1789760790.576793:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769647758764 time 769681501769 state 0x0 ref 1 00000100:00100000:0.0:1789760790.576807:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508477952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760790.576870:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25309184, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:0.0:1789760790.576881:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57004 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.576901:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.576907:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.576915:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57004 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.576932:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57004 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760790.612232:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25309184 -> 25313280 00010000:00010000:1.0:1789760790.612258:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57004 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.612274:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57004 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.612306:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57004 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.612315:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760790.612321:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57004 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.612331:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57004 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.612355:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57004 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.612362:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57004 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25309184->25313279] (req 25309184->25313279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.612494:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508477952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35682us (36334us total) trans 4294973499 rc 0/0 00000100:00100000:1.0:1789760790.612506:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6211 00000100:00100000:1.0:1789760790.612519:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769647758764 time 769681501769 state 0x2 ref 0 00000001:00080000:3.0:1789760790.612607:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973499 is committed 00000100:00100000:0.0:1789760790.615822:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760790.615933:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508478080 00000100:00100000:1.0:1789760790.615947:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508478080 00000100:00100000:1.0:1789760790.616023:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769647758764 time 769681501769 state 0x0 ref 1 00000100:00100000:1.0:1789760790.616042:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769681762185 time 769681501769 state 0x1 ref 1 00000100:00100000:1.0:1789760790.616179:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769681762185 time 769720887812 state 0x0 ref 1 00000100:00100000:1.0:1789760790.616195:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508478080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760790.616265:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25313280, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:1.0:1789760790.616275:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5700b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.616315:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760790.616320:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.616328:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5700b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.616344:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5700b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.658563:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973500 is committed 00002000:00010000:0.0:1789760790.658766:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25313280 -> 25317376 00010000:00010000:0.0:1789760790.658786:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5700b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.658802:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5700b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.658813:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5700b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.658822:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760790.658828:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5700b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.658838:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5700b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.658863:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5700b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.658871:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5700b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25313280->25317375] (req 25313280->25317375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.659026:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508478080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42827us (43399us total) trans 4294973500 rc 0/0 00000100:00100000:0.0:1789760790.659039:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6212 00000100:00100000:0.0:1789760790.659051:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769681762185 time 769720887812 state 0x2 ref 0 00000100:00100000:0.0:1789760790.666484:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760790.666589:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508478208 00000100:00100000:2.0:1789760790.666602:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508478208 00000100:00100000:2.0:1789760790.666688:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769681762185 time 769720887812 state 0x0 ref 1 00000100:00100000:2.0:1789760790.666717:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769721148228 time 769720887812 state 0x1 ref 1 00000100:00100000:2.0:1789760790.666745:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769721148228 time 769771453807 state 0x0 ref 1 00000100:00100000:2.0:1789760790.666762:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508478208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760790.666825:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25317376, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:2.0:1789760790.666837:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57012 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.666857:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760790.666862:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.666870:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57012 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.666887:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57012 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760790.693948:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25317376 -> 25321472 00010000:00010000:2.0:1789760790.693976:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57012 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.693992:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57012 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.694003:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57012 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.694010:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760790.694016:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57012 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.694027:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57012 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.694051:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57012 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.694217:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57012 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25317376->25321471] (req 25317376->25321471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.694400:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508478208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27633us (27923us total) trans 4294973501 rc 0/0 00000100:00100000:2.0:1789760790.694413:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6213 00000100:00100000:2.0:1789760790.694425:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769721148228 time 769771453807 state 0x2 ref 0 00000001:00080000:2.0:1789760790.694468:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973501 is committed 00000100:00100000:3.0:1789760790.697192:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760790.697796:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508478336 00000100:00100000:1.0:1789760790.697810:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508478336 00000100:00100000:1.0:1789760790.697961:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769721148228 time 769771453807 state 0x0 ref 1 00000100:00100000:1.0:1789760790.697983:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769771714223 time 769771453807 state 0x1 ref 1 00000100:00100000:1.0:1789760790.698059:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769771714223 time 769802768297 state 0x0 ref 1 00000100:00100000:1.0:1789760790.698074:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508478336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760790.698144:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25321472, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:1.0:1789760790.698156:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57019 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.698177:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:1.0:1789760790.698182:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.698190:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57019 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.698208:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57019 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.730862:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973502 is committed 00002000:00010000:0.0:1789760790.731097:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25321472 -> 25325568 00010000:00010000:0.0:1789760790.731117:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57019 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.731133:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57019 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.731150:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57019 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.731158:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760790.731164:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57019 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.731174:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57019 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.731200:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57019 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.731209:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57019 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25321472->25325567] (req 25321472->25325567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.731380:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508478336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33301us (34193us total) trans 4294973502 rc 0/0 00000100:00100000:0.0:1789760790.731392:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6214 00000100:00100000:0.0:1789760790.731405:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769771714223 time 769802768297 state 0x2 ref 0 00000100:00100000:2.0:1789760790.734513:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760790.734922:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508478464 00000100:00100000:0.0:1789760790.734937:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508478464 00000100:00100000:0.0:1789760790.735014:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769771714223 time 769802768297 state 0x0 ref 1 00000100:00100000:0.0:1789760790.735035:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769803028713 time 769802768297 state 0x1 ref 1 00000100:00100000:0.0:1789760790.735216:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769803028713 time 769839924434 state 0x0 ref 1 00000100:00100000:0.0:1789760790.735233:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508478464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760790.735497:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25325568, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:0.0:1789760790.735510:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57020 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.735530:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.735535:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.735544:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57020 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.735562:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57020 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760790.775814:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25325568 -> 25329664 00010000:00010000:0.0:1789760790.775843:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57020 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.775858:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57020 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.775868:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57020 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.775875:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760790.775881:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57020 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.775890:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57020 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.775913:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57020 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.775920:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57020 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25325568->25329663] (req 25325568->25329663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.776044:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508478464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40807us (41541us total) trans 4294973503 rc 0/0 00000100:00100000:0.0:1789760790.776066:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6215 00000100:00100000:0.0:1789760790.776079:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769803028713 time 769839924434 state 0x2 ref 0 00000001:00080000:2.0:1789760790.776981:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973503 is committed 00000100:00100000:3.0:1789760790.784750:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760790.785076:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508478592 00000100:00100000:2.0:1789760790.785089:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508478592 00000100:00100000:2.0:1789760790.785161:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769803028713 time 769839924434 state 0x0 ref 1 00000100:00100000:2.0:1789760790.785181:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769840184850 time 769839924434 state 0x1 ref 1 00000100:00100000:2.0:1789760790.785214:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769840184850 time 769889923297 state 0x0 ref 1 00000100:00100000:2.0:1789760790.785232:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508478592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760790.785529:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25329664, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:2.0:1789760790.785542:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57027 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.785562:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:2.0:1789760790.785567:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.785575:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57027 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.785593:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57027 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760790.843864:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25329664 -> 25333760 00010000:00010000:2.0:1789760790.843895:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57027 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.843911:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57027 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.843920:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57027 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.843928:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760790.843934:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57027 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.843945:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57027 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.843970:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57027 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.843978:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57027 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25329664->25333759] (req 25329664->25333759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.844115:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508478592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58878us (59372us total) trans 4294973504 rc 0/0 00000100:00100000:2.0:1789760790.844127:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6216 00000100:00100000:2.0:1789760790.844139:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769840184850 time 769889923297 state 0x2 ref 0 00000001:00080000:1.0:1789760790.845568:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973504 is committed 00000100:00100000:1.0:1789760790.848570:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760790.848728:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508478720 00000100:00100000:0.0:1789760790.848741:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508478720 00000100:00100000:0.0:1789760790.848815:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769840184850 time 769889923297 state 0x0 ref 1 00000100:00100000:0.0:1789760790.848834:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769890183713 time 769889923297 state 0x1 ref 1 00000100:00100000:0.0:1789760790.848878:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769890183713 time 769953587613 state 0x0 ref 1 00000100:00100000:0.0:1789760790.848892:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508478720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760790.848959:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25333760, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:0.0:1789760790.848971:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5702e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.848990:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=769s v=(0 0) 00010000:00010000:0.0:1789760790.848995:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.849003:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5702e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.849019:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5702e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.894447:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973505 is committed 00002000:00010000:3.0:1789760790.894599:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25333760 -> 25337856 00010000:00010000:3.0:1789760790.894759:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5702e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.894776:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5702e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.894786:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5702e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.894795:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760790.894800:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5702e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.894811:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5702e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.894836:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5702e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.894846:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5702e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25333760->25337855] (req 25333760->25337855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.895006:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508478720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46108us (46445us total) trans 4294973505 rc 0/0 00000100:00100000:3.0:1789760790.895033:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6217 00000100:00100000:3.0:1789760790.895047:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769890183713 time 769953587613 state 0x2 ref 0 00000100:00100000:3.0:1789760790.898421:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760790.898786:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508478848 00000100:00100000:0.0:1789760790.898801:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508478848 00000100:00100000:0.0:1789760790.898878:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769890183713 time 769953587613 state 0x0 ref 1 00000100:00100000:0.0:1789760790.898897:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769953848029 time 769953587613 state 0x1 ref 1 00000100:00100000:0.0:1789760790.899114:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769953848029 time 770003822216 state 0x0 ref 1 00000100:00100000:0.0:1789760790.899132:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508478848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760790.899195:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25337856, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:0.0:1789760790.899205:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe57035 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.899223:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760790.899228:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.899237:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe57035 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.899257:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe57035 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.942153:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973506 is committed 00002000:00010000:1.0:1789760790.942374:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25337856 -> 25341952 00010000:00010000:1.0:1789760790.942394:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe57035 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.942408:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe57035 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.942419:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe57035 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.942427:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760790.942433:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe57035 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.942448:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe57035 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.942474:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe57035 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.942483:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe57035 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25337856->25341951] (req 25337856->25341951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.942735:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508478848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 43599us (44322us total) trans 4294973506 rc 0/0 00000100:00100000:1.0:1789760790.942749:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6218 00000100:00100000:1.0:1789760790.942762:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769953848029 time 770003822216 state 0x2 ref 0 00000100:00100000:0.0:1789760790.945821:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760790.945886:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508478976 00000100:00100000:2.0:1789760790.945899:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508478976 00000100:00100000:2.0:1789760790.945975:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 769953848029 time 770003822216 state 0x0 ref 1 00000100:00100000:2.0:1789760790.945995:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770004082632 time 770003822216 state 0x1 ref 1 00000100:00100000:2.0:1789760790.946035:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770004082632 time 770050744891 state 0x0 ref 1 00000100:00100000:2.0:1789760790.946049:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508478976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760790.946217:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25341952, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:2.0:1789760790.946228:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5703c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.946248:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760790.946253:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.946260:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5703c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.946276:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5703c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.971964:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973507 is committed 00002000:00010000:1.0:1789760790.972153:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25341952 -> 25346048 00010000:00010000:1.0:1789760790.972175:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5703c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.972191:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5703c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.972202:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5703c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.972210:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760790.972216:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5703c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.972227:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5703c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.972253:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5703c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.972262:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5703c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25341952->25346047] (req 25341952->25346047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.972431:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508478976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26376us (26613us total) trans 4294973507 rc 0/0 00000100:00100000:1.0:1789760790.972444:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6219 00000100:00100000:1.0:1789760790.972457:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770004082632 time 770050744891 state 0x2 ref 0 00000100:00100000:0.0:1789760790.975481:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760790.975588:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508479104 00000100:00100000:3.0:1789760790.975602:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508479104 00000100:00100000:3.0:1789760790.975686:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770004082632 time 770050744891 state 0x0 ref 1 00000100:00100000:3.0:1789760790.975717:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770051005307 time 770050744891 state 0x1 ref 1 00000100:00100000:3.0:1789760790.975758:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770051005307 time 770080468056 state 0x0 ref 1 00000100:00100000:3.0:1789760790.975772:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508479104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760790.975850:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25346048, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:3.0:1789760790.975862:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe57043 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.975882:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760790.975888:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.975896:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe57043 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.975914:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe57043 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.019996:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973508 is committed 00002000:00010000:1.0:1789760791.020258:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25346048 -> 25350144 00010000:00010000:1.0:1789760791.020279:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe57043 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.020330:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe57043 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.020340:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe57043 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.020350:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760791.020355:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe57043 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.020362:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe57043 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.020382:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe57043 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.020387:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe57043 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25346048->25350143] (req 25346048->25350143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.020510:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508479104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44733us (45036us total) trans 4294973508 rc 0/0 00000100:00100000:1.0:1789760791.020520:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6220 00000100:00100000:1.0:1789760791.020533:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770051005307 time 770080468056 state 0x2 ref 0 00000100:00100000:0.0:1789760791.024938:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760791.026472:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508479232 00000100:00100000:1.0:1789760791.026492:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508479232 00000100:00100000:0.0:1789760791.026656:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770051005307 time 770080468056 state 0x0 ref 1 00000100:00100000:0.0:1789760791.026677:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770080728472 time 770080468056 state 0x1 ref 1 00000100:00100000:0.0:1789760791.026722:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770080728472 time 770131431469 state 0x0 ref 1 00000100:00100000:0.0:1789760791.026737:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508479232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760791.026804:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25350144, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:0.0:1789760791.026815:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5704a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.026836:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760791.026841:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.026850:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5704a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.026867:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5704a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760791.026957:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760791.087692:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973509 is committed 00002000:00010000:0.0:1789760791.088199:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25350144 -> 25354240 00010000:00010000:0.0:1789760791.088219:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5704a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.088233:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5704a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.088243:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5704a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.088251:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760791.088256:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5704a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.088265:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5704a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.088676:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5704a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.088687:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5704a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25350144->25354239] (req 25350144->25354239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.088868:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7efb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508479232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 62127us (63937us total) trans 4294973509 rc 0/0 00000100:00100000:0.0:1789760791.088881:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6221 00000100:00100000:0.0:1789760791.088894:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770080728472 time 770131431469 state 0x2 ref 0 00000100:00100000:0.0:1789760791.092449:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760791.092546:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508479360 00000100:00100000:1.0:1789760791.092560:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508479360 00000100:00100000:1.0:1789760791.092629:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770080728472 time 770131431469 state 0x0 ref 1 00000100:00100000:1.0:1789760791.092647:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770131691885 time 770131431469 state 0x1 ref 1 00000100:00100000:1.0:1789760791.092688:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770131691885 time 770197398248 state 0x0 ref 1 00000100:00100000:1.0:1789760791.092716:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508479360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760791.092782:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25354240, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:1.0:1789760791.092793:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57051 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.092812:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760791.092818:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.092826:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57051 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.092842:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57051 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.151148:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973510 is committed 00002000:00010000:2.0:1789760791.151367:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25354240 -> 25358336 00010000:00010000:2.0:1789760791.151390:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57051 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.151404:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57051 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.151414:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57051 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.151422:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760791.151427:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57051 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.151438:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57051 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.151464:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57051 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.151473:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57051 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25354240->25358335] (req 25354240->25358335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.151590:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508479360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 58883us (59148us total) trans 4294973510 rc 0/0 00000100:00100000:2.0:1789760791.151602:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6222 00000100:00100000:2.0:1789760791.151615:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770131691885 time 770197398248 state 0x2 ref 0 00000100:00100000:1.0:1789760791.158396:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760791.158680:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508479488 00000100:00100000:3.0:1789760791.158697:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508479488 00000100:00100000:3.0:1789760791.158804:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770131691885 time 770197398248 state 0x0 ref 1 00000100:00100000:3.0:1789760791.158827:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770197658664 time 770197398248 state 0x1 ref 1 00000100:00100000:3.0:1789760791.158944:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770197658664 time 770263653362 state 0x0 ref 1 00000100:00100000:3.0:1789760791.158961:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508479488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760791.159029:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25358336, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:3.0:1789760791.159040:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57058 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.159069:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.159074:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.159083:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57058 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.159101:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57058 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760791.206362:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25358336 -> 25362432 00010000:00010000:3.0:1789760791.206389:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57058 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.206405:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57058 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.206415:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57058 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.206423:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760791.206429:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57058 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.206439:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57058 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.206468:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57058 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.206476:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57058 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25358336->25362431] (req 25358336->25362431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.206595:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508479488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47630us (48206us total) trans 4294973511 rc 0/0 00000100:00100000:3.0:1789760791.206608:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6223 00000100:00100000:3.0:1789760791.208304:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770197658664 time 770263653362 state 0x2 ref 0 00000001:00080000:3.0:1789760791.208380:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973511 is committed 00000100:00100000:3.0:1789760791.211723:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760791.211833:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508479616 00000100:00100000:0.0:1789760791.211845:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508479616 00000100:00100000:0.0:1789760791.211918:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770197658664 time 770263653362 state 0x0 ref 1 00000100:00100000:0.0:1789760791.211937:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770263913778 time 770263653362 state 0x1 ref 1 00000100:00100000:0.0:1789760791.211973:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770263913778 time 770316682906 state 0x0 ref 1 00000100:00100000:0.0:1789760791.211987:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508479616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760791.212056:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25362432, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:0.0:1789760791.212069:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5705f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.212089:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760791.212094:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.212101:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5705f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.212120:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5705f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760791.257363:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25362432 -> 25366528 00010000:00010000:1.0:1789760791.257392:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5705f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.257407:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5705f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.257416:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5705f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.257428:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760791.257434:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5705f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.257442:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5705f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.257467:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5705f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.257475:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5705f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25362432->25366527] (req 25362432->25366527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.257749:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508479616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45756us (46048us total) trans 4294973512 rc 0/0 00000100:00100000:1.0:1789760791.257762:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6224 00000100:00100000:1.0:1789760791.257775:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770263913778 time 770316682906 state 0x2 ref 0 00000001:00080000:3.0:1789760791.257918:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973512 is committed 00000100:00100000:0.0:1789760791.262894:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760791.263003:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508479744 00000100:00100000:2.0:1789760791.263014:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508479744 00000100:00100000:2.0:1789760791.263088:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770263913778 time 770316682906 state 0x0 ref 1 00000100:00100000:2.0:1789760791.263109:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770316943322 time 770316682906 state 0x1 ref 1 00000100:00100000:2.0:1789760791.263242:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770316943322 time 770367949598 state 0x0 ref 1 00000100:00100000:2.0:1789760791.263260:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508479744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760791.263359:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25366528, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:2.0:1789760791.263371:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57066 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.263392:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760791.263396:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.263402:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57066 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.263422:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57066 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.292163:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973513 is committed 00002000:00010000:2.0:1789760791.292359:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25366528 -> 25370624 00010000:00010000:2.0:1789760791.292379:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57066 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.292394:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57066 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.292403:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57066 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.292411:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760791.292416:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57066 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.292426:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57066 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.292452:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57066 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.292460:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe57066 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25366528->25370623] (req 25366528->25370623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.292591:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7efb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508479744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29328us (29707us total) trans 4294973513 rc 0/0 00000100:00100000:2.0:1789760791.292603:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6225 00000100:00100000:2.0:1789760791.292615:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770316943322 time 770367949598 state 0x2 ref 0 00010000:00010000:1.0:1789760791.294434:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760791.296192:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760791.296351:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508479872 00000100:00100000:3.0:1789760791.296367:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508479872 00000100:00100000:3.0:1789760791.296440:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770316943322 time 770367949598 state 0x0 ref 1 00000100:00100000:3.0:1789760791.296462:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770368210014 time 770367949598 state 0x1 ref 1 00000100:00100000:3.0:1789760791.296498:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770368210014 time 770401207492 state 0x0 ref 1 00000100:00100000:3.0:1789760791.296512:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508479872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760791.296576:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25370624, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:3.0:1789760791.296587:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5706d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.296608:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.296722:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.296732:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5706d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.296751:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5706d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760791.332680:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25370624 -> 25374720 00010000:00010000:1.0:1789760791.332718:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5706d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.332733:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5706d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.332748:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5706d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.332756:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760791.332761:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5706d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.332771:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5706d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.332797:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5706d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.332806:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe5706d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25370624->25374719] (req 25370624->25374719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.332926:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508479872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36409us (36740us total) trans 4294973514 rc 0/0 00000100:00100000:1.0:1789760791.332938:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6226 00000100:00100000:1.0:1789760791.332950:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770368210014 time 770401207492 state 0x2 ref 0 00000001:00080000:1.0:1789760791.333021:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973514 is committed 00000100:00100000:3.0:1789760791.336943:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760791.337086:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508480000 00000100:00100000:1.0:1789760791.337100:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508480000 00000100:00100000:1.0:1789760791.337175:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770368210014 time 770401207492 state 0x0 ref 1 00000100:00100000:1.0:1789760791.337197:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770401467908 time 770401207492 state 0x1 ref 1 00000100:00100000:1.0:1789760791.337335:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770401467908 time 770442043494 state 0x0 ref 1 00000100:00100000:1.0:1789760791.337351:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508480000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760791.337415:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25374720, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:1.0:1789760791.337427:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57074 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.337448:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:1.0:1789760791.337453:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.337471:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57074 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.337488:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57074 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.378302:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973515 is committed 00002000:00010000:2.0:1789760791.378561:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25374720 -> 25378816 00010000:00010000:2.0:1789760791.378581:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57074 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.378597:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57074 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.378608:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57074 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.378616:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760791.378623:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57074 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.378633:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57074 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.378658:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57074 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.378672:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57074 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25374720->25378815] (req 25374720->25378815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.378944:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508480000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41589us (42010us total) trans 4294973515 rc 0/0 00000100:00100000:2.0:1789760791.378959:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6227 00000100:00100000:2.0:1789760791.378972:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770401467908 time 770442043494 state 0x2 ref 0 00000100:00100000:3.0:1789760791.383696:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760791.383809:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508480128 00000100:00100000:2.0:1789760791.383823:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508480128 00000100:00100000:2.0:1789760791.383900:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770401467908 time 770442043494 state 0x0 ref 1 00000100:00100000:2.0:1789760791.383919:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770442303910 time 770442043494 state 0x1 ref 1 00000100:00100000:2.0:1789760791.383949:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770442303910 time 770488659636 state 0x0 ref 1 00000100:00100000:2.0:1789760791.383964:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508480128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760791.384030:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25378816, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:2.0:1789760791.384040:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5707b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.384058:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760791.384063:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.384070:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5707b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.384091:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5707b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.425235:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973516 is committed 00002000:00010000:2.0:1789760791.425333:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25378816 -> 25382912 00010000:00010000:2.0:1789760791.425358:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5707b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.425374:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5707b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.425383:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5707b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.425392:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760791.425397:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5707b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.425408:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5707b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.425435:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5707b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.425441:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5707b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25378816->25382911] (req 25378816->25382911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.425844:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508480128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41875us (42155us total) trans 4294973516 rc 0/0 00000100:00100000:2.0:1789760791.425858:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6228 00000100:00100000:2.0:1789760791.425870:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770442303910 time 770488659636 state 0x2 ref 0 00000100:00100000:0.0:1789760791.430170:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760791.430238:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508480256 00000100:00100000:1.0:1789760791.430250:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508480256 00000100:00100000:1.0:1789760791.430488:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770442303910 time 770488659636 state 0x0 ref 1 00000100:00100000:1.0:1789760791.430508:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770488920052 time 770488659636 state 0x1 ref 1 00000100:00100000:0.0:1789760791.430679:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770488920052 time 770535386335 state 0x0 ref 1 00000100:00100000:0.0:1789760791.430696:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508480256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760791.431402:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25382912, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:0.0:1789760791.431416:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57082 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.431434:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760791.431440:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.431447:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57082 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.431464:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57082 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.466569:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973517 is committed 00002000:00010000:3.0:1789760791.466788:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25382912 -> 25387008 00010000:00010000:3.0:1789760791.466816:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57082 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.466832:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57082 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.466842:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57082 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.466849:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760791.466855:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57082 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.466865:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57082 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.466890:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57082 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.466898:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57082 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25382912->25387007] (req 25382912->25387007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.467015:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7ef8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508480256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36315us (36852us total) trans 4294973517 rc 0/0 00000100:00100000:3.0:1789760791.467027:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6229 00000100:00100000:3.0:1789760791.467040:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770488920052 time 770535386335 state 0x2 ref 0 00000100:00100000:3.0:1789760791.473238:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760791.473367:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508480384 00000100:00100000:2.0:1789760791.473380:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508480384 00000100:00100000:2.0:1789760791.473454:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770488920052 time 770535386335 state 0x0 ref 1 00000100:00100000:2.0:1789760791.473473:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770535646751 time 770535386335 state 0x1 ref 1 00000100:00100000:2.0:1789760791.473508:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770535646751 time 770578218104 state 0x0 ref 1 00000100:00100000:2.0:1789760791.473522:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508480384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760791.473589:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25387008, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0xc807 00010000:00010000:2.0:1789760791.473600:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57089 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.473689:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:2.0:1789760791.473695:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.473710:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57089 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.473917:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57089 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.532649:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973518 is committed 00002000:00010000:1.0:1789760791.532938:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25387008 -> 25391104 00002000:00010000:1.0:1789760791.532944:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760791 -> 1789760792 00002000:00010000:1.0:1789760791.532948:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760791 -> 1789760792 00010000:00010000:1.0:1789760791.532963:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57089 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.532976:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57089 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.532984:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57089 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.532991:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760791.532996:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57089 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.533006:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57089 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.533031:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57089 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.533040:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57089 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25387008->25391103] (req 25387008->25391103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.533175:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508480384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 59647us (59943us total) trans 4294973518 rc 0/0 00000100:00100000:1.0:1789760791.533187:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6230 00000100:00100000:1.0:1789760791.533199:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770535646751 time 770578218104 state 0x2 ref 0 00000100:00100000:2.0:1789760791.536438:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760791.536543:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508480512 00000100:00100000:0.0:1789760791.536608:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508480512 00000100:00100000:0.0:1789760791.536870:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770535646751 time 770578218104 state 0x0 ref 1 00000100:00100000:0.0:1789760791.536890:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770578478520 time 770578218104 state 0x1 ref 1 00000100:00100000:0.0:1789760791.536923:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770578478520 time 770641633737 state 0x0 ref 1 00000100:00100000:0.0:1789760791.536938:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508480512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760791.536998:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25391104, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:0.0:1789760791.537008:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57090 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.537027:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:0.0:1789760791.537031:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.537039:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57090 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.537055:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57090 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.609117:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973519 is committed 00002000:00010000:0.0:1789760791.609180:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25391104 -> 25395200 00010000:00010000:0.0:1789760791.609199:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57090 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.609211:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57090 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.609218:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57090 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.609222:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760791.609227:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57090 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.609235:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57090 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.609256:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57090 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.609261:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57090 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25391104->25395199] (req 25391104->25395199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.609505:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508480512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 72562us (73073us total) trans 4294973519 rc 0/0 00000100:00100000:0.0:1789760791.609517:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6231 00000100:00100000:0.0:1789760791.609531:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770578478520 time 770641633737 state 0x2 ref 0 00000100:00100000:2.0:1789760791.614792:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760791.616114:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508480640 00000100:00100000:3.0:1789760791.616134:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508480640 00000100:00100000:3.0:1789760791.616230:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770578478520 time 770641633737 state 0x0 ref 1 00000100:00100000:3.0:1789760791.616250:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770641894153 time 770641633737 state 0x1 ref 1 00000100:00100000:3.0:1789760791.616276:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770641894153 time 770720986390 state 0x0 ref 1 00000100:00100000:3.0:1789760791.616317:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508480640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760791.616384:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25395200, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:3.0:1789760791.616395:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57097 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.616415:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.616421:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.616428:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57097 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.616446:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57097 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.619946:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760791.619989:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760791.620008:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760791.620029:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760791.620055:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760791.620094:0:6459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508480768 00000100:00100000:2.0:1789760791.620106:0:6459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508480768 00000100:00100000:2.0:1789760791.620166:0:6459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 491 00000100:00100000:2.0:1789760791.620178:0:6459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988cab050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2409:x1876699508480768:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760791.620271:0:6459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988cab050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2409:x1876699508480768:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 95us (337us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760791.620281:0:6459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 491 00000100:00100000:2.0:1789760791.620336:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508481024 00000100:00100000:2.0:1789760791.620340:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508481024 00000100:00100000:2.0:1789760791.620380:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508481280 00000100:00100000:2.0:1789760791.620383:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508481280 00000100:00100000:2.0:1789760791.620404:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508480896 00000100:00100000:2.0:1789760791.620406:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508480896 00000100:00100000:2.0:1789760791.620433:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508481152 00000100:00100000:2.0:1789760791.620436:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508481152 00000100:00100000:2.0:1789760791.620453:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 989 00000100:00100000:2.0:1789760791.620815:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508481024:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760791.620861:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508481024:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 51us (871us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760791.620873:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 989 00000100:00100000:2.0:1789760791.620888:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 990 00000100:00100000:2.0:1789760791.620895:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508481280:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760791.620921:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508481280:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 26us (911us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760791.620928:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 990 00000100:00100000:2.0:1789760791.620940:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 991 00000100:00100000:2.0:1789760791.620946:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2409:x1876699508480896:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760791.620970:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2409:x1876699508480896:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 25us (941us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760791.620976:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 991 00000100:00100000:2.0:1789760791.620988:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 992 00000100:00100000:2.0:1789760791.620994:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508481152:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760791.621022:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508481152:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 28us (966us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760791.621028:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 992 00000100:00080000:2.0:1789760791.621054:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760791.621057:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00002000:00010000:3.0:1789760791.661971:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25395200 -> 25399296 00010000:00010000:3.0:1789760791.662003:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57097 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.662019:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57097 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.662030:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57097 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.662038:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760791.662044:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57097 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.662065:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57097 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.662091:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57097 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.662101:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57097 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25395200->25399295] (req 25395200->25399295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.662228:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508480640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45934us (47444us total) trans 4294973520 rc 0/0 00000100:00100000:3.0:1789760791.662241:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6232 00000100:00100000:3.0:1789760791.662254:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770641894153 time 770720986390 state 0x2 ref 0 00000001:00080000:0.0:1789760791.662593:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973520 is committed 00000100:00100000:2.0:1789760791.665855:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760791.665964:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508481408 00000100:00100000:3.0:1789760791.665978:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508481408 00000100:00100000:3.0:1789760791.666062:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770641894153 time 770720986390 state 0x0 ref 1 00000100:00100000:3.0:1789760791.666081:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770721246806 time 770720986390 state 0x1 ref 1 00000100:00100000:3.0:1789760791.666113:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770721246806 time 770770823219 state 0x0 ref 1 00000100:00100000:3.0:1789760791.666127:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508481408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760791.666193:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25399296, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:3.0:1789760791.666203:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5709e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.666223:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.666228:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.666236:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5709e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.666253:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5709e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760791.705783:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25399296 -> 25403392 00010000:00010000:3.0:1789760791.705810:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5709e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.705827:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5709e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.705842:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5709e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.705851:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760791.705858:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5709e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.705867:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5709e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.705894:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5709e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.705902:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5709e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25399296->25403391] (req 25399296->25403391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.706033:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508481408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39901us (40185us total) trans 4294973521 rc 0/0 00000100:00100000:3.0:1789760791.706045:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6233 00000100:00100000:3.0:1789760791.706066:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770721246806 time 770770823219 state 0x2 ref 0 00000001:00080000:0.0:1789760791.706149:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973521 is committed 00000100:00100000:2.0:1789760791.709119:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760791.709226:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508481536 00000100:00100000:3.0:1789760791.709242:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508481536 00000100:00100000:3.0:1789760791.709352:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770721246806 time 770770823219 state 0x0 ref 1 00000100:00100000:3.0:1789760791.709371:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770771083635 time 770770823219 state 0x1 ref 1 00000100:00100000:3.0:1789760791.709399:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770771083635 time 770814109047 state 0x0 ref 1 00000100:00100000:3.0:1789760791.709413:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508481536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760791.709481:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25403392, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:3.0:1789760791.709492:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.709513:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.709519:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.709526:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.709543:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760791.741531:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25403392 -> 25407488 00010000:00010000:3.0:1789760791.741561:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.741576:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.741586:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.741594:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760791.741600:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.741622:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.741649:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.741656:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25403392->25407487] (req 25403392->25407487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.741786:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508481536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32368us (32675us total) trans 4294973522 rc 0/0 00000100:00100000:3.0:1789760791.741799:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6234 00000100:00100000:3.0:1789760791.741811:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770771083635 time 770814109047 state 0x2 ref 0 00000001:00080000:1.0:1789760791.744378:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973522 is committed 00000100:00100000:1.0:1789760791.745174:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760791.745260:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508481664 00000100:00100000:3.0:1789760791.745273:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508481664 00000100:00100000:3.0:1789760791.745381:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770771083635 time 770814109047 state 0x0 ref 1 00000100:00100000:3.0:1789760791.745401:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770814369463 time 770814109047 state 0x1 ref 1 00000100:00100000:3.0:1789760791.745434:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770814369463 time 770850144132 state 0x0 ref 1 00000100:00100000:3.0:1789760791.745447:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508481664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760791.745509:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25407488, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:3.0:1789760791.745520:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.745540:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.745545:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.745552:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.745568:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.778721:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973523 is committed 00002000:00010000:3.0:1789760791.778866:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25407488 -> 25411584 00010000:00010000:3.0:1789760791.778890:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.778906:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.778917:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.778925:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760791.778931:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.778942:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.778975:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.778983:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25407488->25411583] (req 25407488->25411583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.779179:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508481664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33725us (34007us total) trans 4294973523 rc 0/0 00000100:00100000:3.0:1789760791.779192:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6235 00000100:00100000:3.0:1789760791.779204:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770814369463 time 770850144132 state 0x2 ref 0 00000100:00100000:2.0:1789760791.782414:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760791.782520:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508481792 00000100:00100000:3.0:1789760791.782533:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508481792 00000100:00100000:3.0:1789760791.782605:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770814369463 time 770850144132 state 0x0 ref 1 00000100:00100000:3.0:1789760791.782695:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770850404548 time 770850144132 state 0x1 ref 1 00000100:00100000:3.0:1789760791.782789:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770850404548 time 770887497788 state 0x0 ref 1 00000100:00100000:3.0:1789760791.782804:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508481792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760791.782871:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25411584, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:3.0:1789760791.782883:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.782904:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.782910:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.782918:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.782934:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.819983:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973524 is committed 00002000:00010000:1.0:1789760791.820182:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25411584 -> 25415680 00010000:00010000:1.0:1789760791.820202:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.820218:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.820227:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.820235:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760791.820241:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.820251:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.820275:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.820999:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe570b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25411584->25415679] (req 25411584->25415679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.821323:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508481792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38513us (38914us total) trans 4294973524 rc 0/0 00000100:00100000:2.0:1789760791.821338:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6236 00000100:00100000:2.0:1789760791.821351:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770850404548 time 770887497788 state 0x2 ref 0 00000100:00100000:0.0:1789760791.825181:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760791.825262:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508481920 00000100:00100000:3.0:1789760791.825276:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508481920 00000100:00100000:3.0:1789760791.825602:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770850404548 time 770887497788 state 0x0 ref 1 00000100:00100000:3.0:1789760791.825721:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770887758204 time 770887497788 state 0x1 ref 1 00000100:00100000:3.0:1789760791.825785:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770887758204 time 770930493848 state 0x0 ref 1 00000100:00100000:3.0:1789760791.825800:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508481920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760791.825870:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25415680, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:3.0:1789760791.825882:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.825903:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.825909:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.825917:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.825935:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760791.852733:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25415680 -> 25419776 00010000:00010000:3.0:1789760791.852762:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.852778:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.852788:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.852795:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760791.852800:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.852811:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.852836:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.852845:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25415680->25419775] (req 25415680->25419775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.852963:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508481920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27158us (27789us total) trans 4294973525 rc 0/0 00000100:00100000:3.0:1789760791.852980:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6237 00000100:00100000:3.0:1789760791.852994:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770887758204 time 770930493848 state 0x2 ref 0 00000001:00080000:3.0:1789760791.853042:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973525 is committed 00000100:00100000:0.0:1789760791.856855:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760791.856961:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508482048 00000100:00100000:3.0:1789760791.856974:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508482048 00000100:00100000:3.0:1789760791.857107:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770887758204 time 770930493848 state 0x0 ref 1 00000100:00100000:3.0:1789760791.857128:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770930754264 time 770930493848 state 0x1 ref 1 00000100:00100000:3.0:1789760791.857207:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770930754264 time 770961916575 state 0x0 ref 1 00000100:00100000:3.0:1789760791.857221:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508482048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760791.857308:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25419776, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:3.0:1789760791.857320:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe570c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.857340:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=770s v=(0 0) 00010000:00010000:3.0:1789760791.857345:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.857353:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe570c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.857372:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe570c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.892375:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973526 is committed 00002000:00010000:3.0:1789760791.892511:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25419776 -> 25423872 00010000:00010000:3.0:1789760791.892532:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe570c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.892547:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe570c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.892559:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe570c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.892568:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760791.892573:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe570c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.892584:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe570c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.892622:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe570c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.892632:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe570c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25419776->25423871] (req 25419776->25423871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.892768:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508482048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35541us (35921us total) trans 4294973526 rc 0/0 00000100:00100000:3.0:1789760791.892780:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6238 00000100:00100000:3.0:1789760791.892793:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770930754264 time 770961916575 state 0x2 ref 0 00000100:00100000:3.0:1789760791.895772:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760791.896143:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508482176 00000100:00100000:2.0:1789760791.896157:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508482176 00000100:00100000:2.0:1789760791.896232:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770930754264 time 770961916575 state 0x0 ref 1 00000100:00100000:2.0:1789760791.896251:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770962176991 time 770961916575 state 0x1 ref 1 00000100:00100000:2.0:1789760791.896403:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770962176991 time 771001111579 state 0x0 ref 1 00000100:00100000:2.0:1789760791.896418:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508482176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760791.896480:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25423872, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:2.0:1789760791.896492:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe570c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.896511:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760791.896516:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.896523:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe570c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.896543:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe570c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760791.938496:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25423872 -> 25427968 00010000:00010000:2.0:1789760791.938525:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe570c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.938547:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe570c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.938557:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe570c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.938566:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760791.938571:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe570c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.938581:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe570c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.938618:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe570c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.938628:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe570c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25423872->25427967] (req 25423872->25427967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.938767:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508482176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42346us (43002us total) trans 4294973527 rc 0/0 00000100:00100000:2.0:1789760791.938779:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6239 00000100:00100000:2.0:1789760791.938792:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770962176991 time 771001111579 state 0x2 ref 0 00000001:00080000:3.0:1789760791.939230:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973527 is committed 00000100:00100000:2.0:1789760791.941874:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760791.942012:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508482304 00000100:00100000:1.0:1789760791.942025:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508482304 00000100:00100000:1.0:1789760791.942102:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 770962176991 time 771001111579 state 0x0 ref 1 00000100:00100000:1.0:1789760791.942121:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771001371995 time 771001111579 state 0x1 ref 1 00000100:00100000:1.0:1789760791.942150:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771001371995 time 771046860578 state 0x0 ref 1 00000100:00100000:1.0:1789760791.942161:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508482304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760791.942220:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25427968, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:1.0:1789760791.942231:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.942250:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760791.942255:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.942262:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.942277:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.988793:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973528 is committed 00002000:00010000:1.0:1789760791.988924:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25427968 -> 25432064 00010000:00010000:1.0:1789760791.988943:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.988958:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.988973:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.988981:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760791.988987:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.988998:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.989022:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.989030:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25427968->25432063] (req 25427968->25432063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.989162:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508482304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46995us (47295us total) trans 4294973528 rc 0/0 00000100:00100000:1.0:1789760791.989175:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6240 00000100:00100000:1.0:1789760791.989187:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771001371995 time 771046860578 state 0x2 ref 0 00000100:00100000:0.0:1789760791.994001:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760791.994074:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508482432 00000100:00100000:3.0:1789760791.994089:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508482432 00000100:00100000:3.0:1789760791.994166:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771001371995 time 771046860578 state 0x0 ref 1 00000100:00100000:3.0:1789760791.994195:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771047120994 time 771046860578 state 0x1 ref 1 00000100:00100000:3.0:1789760791.994224:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771047120994 time 771098934663 state 0x0 ref 1 00000100:00100000:3.0:1789760791.994239:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c1fd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508482432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760791.994327:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25432064, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:3.0:1789760791.994339:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.994358:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760791.994363:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.994371:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.994392:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.029720:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973529 is committed 00002000:00010000:3.0:1789760792.029832:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25432064 -> 25436160 00010000:00010000:3.0:1789760792.029855:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.029876:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.029887:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.029895:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760792.029901:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.029912:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.029938:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.029947:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25432064->25436159] (req 25432064->25436159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.030173:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c1fd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508482432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35929us (36178us total) trans 4294973529 rc 0/0 00000100:00100000:3.0:1789760792.030186:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6241 00000100:00100000:3.0:1789760792.030199:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771047120994 time 771098934663 state 0x2 ref 0 00000100:00100000:3.0:1789760792.034242:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760792.035011:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508482560 00000100:00100000:0.0:1789760792.035027:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508482560 00000100:00100000:0.0:1789760792.035111:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771047120994 time 771098934663 state 0x0 ref 1 00000100:00100000:0.0:1789760792.035132:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771099195079 time 771098934663 state 0x1 ref 1 00000100:00100000:0.0:1789760792.035212:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771099195079 time 771139921399 state 0x0 ref 1 00000100:00100000:0.0:1789760792.035231:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508482560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760792.035309:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25436160, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:0.0:1789760792.035320:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe570dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.035338:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760792.035342:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.035350:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe570dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.035367:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe570dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.078730:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973530 is committed 00002000:00010000:2.0:1789760792.079145:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25436160 -> 25440256 00010000:00010000:2.0:1789760792.079167:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe570dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.079183:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe570dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.079192:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe570dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.079198:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760792.079203:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe570dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.079211:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe570dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.079236:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe570dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.079245:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe570dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25436160->25440255] (req 25436160->25440255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.083107:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508482560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47857us (48858us total) trans 4294973530 rc 0/0 00000100:00100000:3.0:1789760792.083129:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6242 00000100:00100000:3.0:1789760792.083143:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771099195079 time 771139921399 state 0x2 ref 0 00000100:00100000:3.0:1789760792.088501:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760792.088618:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508482688 00000100:00100000:1.0:1789760792.088630:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508482688 00000100:00100000:1.0:1789760792.088710:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771099195079 time 771139921399 state 0x0 ref 1 00000100:00100000:1.0:1789760792.088729:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771140181815 time 771139921399 state 0x1 ref 1 00000100:00100000:1.0:1789760792.088773:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771140181815 time 771193483142 state 0x0 ref 1 00000100:00100000:1.0:1789760792.088787:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508482688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760792.088852:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25440256, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:1.0:1789760792.088864:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.088883:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760792.088890:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.088898:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.088916:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.118731:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973531 is committed 00002000:00010000:2.0:1789760792.118868:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25440256 -> 25444352 00010000:00010000:2.0:1789760792.118889:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.118910:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.118919:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.118926:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760792.118932:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.118942:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.118966:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.118974:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe570e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25440256->25444351] (req 25440256->25444351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.119106:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508482688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30315us (30615us total) trans 4294973531 rc 0/0 00000100:00100000:2.0:1789760792.119119:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6243 00000100:00100000:2.0:1789760792.119131:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771140181815 time 771193483142 state 0x2 ref 0 00000100:00100000:0.0:1789760792.124658:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760792.124779:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508482816 00000100:00100000:3.0:1789760792.124793:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508482816 00000100:00100000:3.0:1789760792.124876:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771140181815 time 771193483142 state 0x0 ref 1 00000100:00100000:3.0:1789760792.124898:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771193743558 time 771193483142 state 0x1 ref 1 00000100:00100000:3.0:1789760792.124938:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771193743558 time 771229647715 state 0x0 ref 1 00000100:00100000:3.0:1789760792.124954:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508482816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760792.125023:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25444352, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:3.0:1789760792.125035:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.125057:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.125063:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.125071:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.125087:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.152653:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973532 is committed 00002000:00010000:3.0:1789760792.152980:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25444352 -> 25448448 00010000:00010000:3.0:1789760792.153000:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.153013:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.153022:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.153029:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760792.153035:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.153147:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.153176:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.153184:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe570eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25444352->25448447] (req 25444352->25448447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.153390:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508482816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28432us (28741us total) trans 4294973532 rc 0/0 00000100:00100000:3.0:1789760792.153407:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6244 00000100:00100000:3.0:1789760792.153419:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771193743558 time 771229647715 state 0x2 ref 0 00000100:00100000:3.0:1789760792.156796:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760792.156935:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508482944 00000100:00100000:0.0:1789760792.156949:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508482944 00000100:00100000:0.0:1789760792.157023:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771193743558 time 771229647715 state 0x0 ref 1 00000100:00100000:0.0:1789760792.157109:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771229908131 time 771229647715 state 0x1 ref 1 00000100:00100000:0.0:1789760792.157194:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771229908131 time 771261903766 state 0x0 ref 1 00000100:00100000:0.0:1789760792.157210:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508482944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760792.157274:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25448448, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:0.0:1789760792.157328:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe570f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.157348:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760792.157354:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.157361:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe570f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.157379:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe570f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.188807:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973533 is committed 00002000:00010000:0.0:1789760792.188940:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25448448 -> 25452544 00010000:00010000:0.0:1789760792.188960:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe570f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.188975:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe570f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.188984:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe570f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.188991:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760792.188997:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe570f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.189006:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe570f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.189038:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe570f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.189115:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe570f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25448448->25452543] (req 25448448->25452543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.189326:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508482944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32071us (32498us total) trans 4294973533 rc 0/0 00000100:00100000:0.0:1789760792.189339:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6245 00000100:00100000:0.0:1789760792.189353:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771229908131 time 771261903766 state 0x2 ref 0 00000100:00100000:2.0:1789760792.193361:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760792.193510:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508483072 00000100:00100000:1.0:1789760792.193525:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508483072 00000100:00100000:1.0:1789760792.193602:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771229908131 time 771261903766 state 0x0 ref 1 00000100:00100000:1.0:1789760792.193634:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771262164182 time 771261903766 state 0x1 ref 1 00000100:00100000:1.0:1789760792.193668:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771262164182 time 771298378936 state 0x0 ref 1 00000100:00100000:1.0:1789760792.193682:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508483072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760792.193759:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25452544, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:1.0:1789760792.193769:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe570f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.193789:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760792.193794:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.193802:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe570f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.193819:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe570f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.227253:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973534 is committed 00002000:00010000:3.0:1789760792.227421:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25452544 -> 25456640 00010000:00010000:3.0:1789760792.227444:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe570f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.227461:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe570f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.227471:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe570f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.227480:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760792.227487:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe570f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.227498:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe570f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.227523:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe570f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.227533:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe570f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25452544->25456639] (req 25452544->25456639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.227674:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508483072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33985us (34319us total) trans 4294973534 rc 0/0 00000100:00100000:3.0:1789760792.227686:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6246 00000100:00100000:3.0:1789760792.227709:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771262164182 time 771298378936 state 0x2 ref 0 00000100:00100000:1.0:1789760792.231584:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760792.231679:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508483200 00000100:00100000:3.0:1789760792.231702:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508483200 00000100:00100000:3.0:1789760792.231778:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771262164182 time 771298378936 state 0x0 ref 1 00000100:00100000:3.0:1789760792.231800:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771298639352 time 771298378936 state 0x1 ref 1 00000100:00100000:3.0:1789760792.231837:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771298639352 time 771336547097 state 0x0 ref 1 00000100:00100000:3.0:1789760792.231850:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508483200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760792.231916:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25456640, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:3.0:1789760792.231927:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57100 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.231947:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.231951:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.231959:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57100 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.231976:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57100 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.285971:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973535 is committed 00002000:00010000:0.0:1789760792.286112:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25456640 -> 25460736 00010000:00010000:0.0:1789760792.286144:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57100 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.286160:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57100 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.286171:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57100 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.286179:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760792.286185:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57100 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.286201:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57100 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.286230:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57100 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.286239:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57100 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25456640->25460735] (req 25456640->25460735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.286551:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888130380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508483200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54694us (54976us total) trans 4294973535 rc 0/0 00000100:00100000:0.0:1789760792.286847:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6247 00000100:00100000:0.0:1789760792.286863:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771298639352 time 771336547097 state 0x2 ref 0 00000100:00100000:0.0:1789760792.289453:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760792.290977:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508483328 00000100:00100000:3.0:1789760792.290997:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508483328 00000100:00100000:3.0:1789760792.291140:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771298639352 time 771336547097 state 0x0 ref 1 00000100:00100000:3.0:1789760792.291160:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771336807513 time 771336547097 state 0x1 ref 1 00000100:00100000:3.0:1789760792.291197:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771336807513 time 771395907456 state 0x0 ref 1 00000100:00100000:3.0:1789760792.291213:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508483328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760792.291273:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25460736, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:3.0:1789760792.291311:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57107 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.291331:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.291337:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.291345:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57107 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.291362:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57107 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.318799:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760792.327405:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25460736 -> 25464832 00010000:00010000:0.0:1789760792.327432:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57107 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.327447:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57107 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.327458:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57107 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.327472:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760792.327478:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57107 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.327489:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57107 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.327513:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57107 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.327521:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57107 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25460736->25464831] (req 25460736->25464831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.327798:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508483328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36581us (38351us total) trans 4294973536 rc 0/0 00000001:00080000:1.0:1789760792.327810:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973536 is committed 00000100:00100000:0.0:1789760792.327812:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6248 00000100:00100000:0.0:1789760792.327827:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771336807513 time 771395907456 state 0x2 ref 0 00000100:00100000:3.0:1789760792.336472:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760792.336816:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508483456 00000100:00100000:1.0:1789760792.336829:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508483456 00000100:00100000:1.0:1789760792.336907:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771336807513 time 771395907456 state 0x0 ref 1 00000100:00100000:1.0:1789760792.336926:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771396167872 time 771395907456 state 0x1 ref 1 00000100:00100000:1.0:1789760792.336964:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771396167872 time 771441674515 state 0x0 ref 1 00000100:00100000:1.0:1789760792.336979:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508483456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760792.337040:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25464832, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:1.0:1789760792.337052:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5710e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.337070:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760792.337075:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.337082:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5710e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.337099:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5710e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.408542:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973537 is committed 00002000:00010000:3.0:1789760792.408759:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25464832 -> 25468928 00010000:00010000:3.0:1789760792.408779:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5710e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.408796:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5710e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.408807:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5710e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.408816:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760792.408822:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5710e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.408830:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5710e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.408856:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5710e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.408865:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe5710e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25464832->25468927] (req 25464832->25468927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.408986:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508483456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 72002us (72522us total) trans 4294973537 rc 0/0 00000100:00100000:3.0:1789760792.408998:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6249 00000100:00100000:3.0:1789760792.409010:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771396167872 time 771441674515 state 0x2 ref 0 00000100:00100000:2.0:1789760792.413731:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760792.414938:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508483584 00000100:00100000:0.0:1789760792.414958:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508483584 00000100:00100000:0.0:1789760792.415038:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771396167872 time 771441674515 state 0x0 ref 1 00000100:00100000:0.0:1789760792.415058:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771441934931 time 771441674515 state 0x1 ref 1 00000100:00100000:0.0:1789760792.415190:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771441934931 time 771519898043 state 0x0 ref 1 00000100:00100000:0.0:1789760792.415209:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508483584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760792.415279:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25468928, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:0.0:1789760792.415339:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe57115 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.415360:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760792.415365:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.415372:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe57115 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.415389:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe57115 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.446676:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7340380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534091648:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.446780:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7340a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3473:1876699534091520:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.447006:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760792.447074:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04987255880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534092160:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.447107:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7340e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3475:1876699534091776:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.447218:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534091648 00000100:00100000:3.0:1789760792.447227:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534091648 00000100:00100000:0.0:1789760792.447274:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760792.447325:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0499b66df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3472:1876699534091392:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.447336:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:3.0:1789760792.447350:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534091648:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.447354:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04987256d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3473:1876699534092032:0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.447395:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.447412:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760792.447500:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049b7342680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534091904:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.447954:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3474:x1876699534091648:12345-0@lo:400:kworker.0 Request processed in 495us (835us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760792.447964:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00100000:1.0:1789760792.447991:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534091520 00000100:00100000:0.0:1789760792.447996:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04987255f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3473:1876699534092544:0@lo:400:kworker.0 00000100:00100000:1.0:1789760792.447999:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534091520 00000100:00100000:3.0:1789760792.448041:0:6460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534091392 00000100:00100000:3.0:1789760792.448046:0:6460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534091392 00000100:00100000:0.0:1789760792.448057:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760792.448090:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 492 00000100:00100000:0.0:1789760792.448094:0:3473:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888131c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534093056:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760792.448102:0:6460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988ca8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3472:x1876699534091392:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.448128:0:3473:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.448162:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534092032 00000100:00100000:0.0:1789760792.448166:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534092032 00000100:00100000:0.0:1789760792.448336:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534092544 00000100:00100000:0.0:1789760792.448342:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534092544 00000100:00100000:0.0:1789760792.448369:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 993 00000100:00100000:0.0:1789760792.448377:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3473:x1876699534091520:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.448420:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3473:x1876699534091520:12345-0@lo:400:kworker.0 Request processed in 44us (1147us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.448427:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 993 00000100:00100000:0.0:1789760792.448442:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 994 00000100:00100000:0.0:1789760792.448448:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3473:x1876699534092032:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.448477:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3473:x1876699534092032:12345-0@lo:400:kworker.0 Request processed in 29us (1068us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.448483:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 994 00000100:00100000:0.0:1789760792.448497:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 995 00000100:00100000:0.0:1789760792.448503:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3473:x1876699534092544:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.448528:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3473:x1876699534092544:12345-0@lo:400:kworker.0 Request processed in 26us (473us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.448534:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 995 00000100:00080000:0.0:1789760792.448578:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760792.448624:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7340a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3473:1876699534091520:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.448637:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04987256d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3473:1876699534092032:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.448645:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04987255f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3473:1876699534092544:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.448732:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.448759:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04987255180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534092288:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.448788:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.448807:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04984e7ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534092800:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.448830:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.448865:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048851d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534093312:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760792.448888:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.448909:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760792.448927:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04984e7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534093184:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760792.448963:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760792.448974:0:6460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988ca8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3472:x1876699534091392:12345-0@lo:400:kworker.0 Request processed in 872us (1577us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760792.448985:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 492 00000100:00100000:0.0:1789760792.448999:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534092288 00000100:00100000:0.0:1789760792.449005:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534092288 00000100:00100000:3.0:1789760792.449027:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534093056 00000100:00100000:3.0:1789760792.449033:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534093056 00000100:00100000:0.0:1789760792.449048:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534092800 00000100:00100000:0.0:1789760792.449054:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534092800 00000100:00100000:3.0:1789760792.449061:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534093312 00000100:00100000:3.0:1789760792.449065:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534093312 00000100:00100000:0.0:1789760792.449086:0:8102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:3.0:1789760792.449086:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534093184 00000100:00100000:3.0:1789760792.449090:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534093184 00000100:00100000:0.0:1789760792.449098:0:8102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534092288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.449106:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:3.0:1789760792.449115:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534093056:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760792.449141:0:8102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534092288:12345-0@lo:400:kworker.0 Request processed in 47us (351us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.449149:0:8102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00100000:0.0:1789760792.449162:0:8102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 253 00000100:00100000:0.0:1789760792.449168:0:8102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534092800:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.449206:0:8102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534092800:12345-0@lo:400:kworker.0 Request processed in 39us (374us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.449211:0:8102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 253 00000100:00100000:0.0:1789760792.449234:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:0.0:1789760792.449241:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534093312:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760792.449273:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760792.449330:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534093312:12345-0@lo:13:osp-pre-1-0.0 Request processed in 90us (441us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.449337:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:0.0:1789760792.449349:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:0.0:1789760792.449355:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534093184:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1789760792.449365:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789760792.449389:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3475:x1876699534093184:12345-0@lo:13:osp-pre-1-1.0 Request processed in 34us (426us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.449394:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00080000:0.0:1789760792.449408:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760792.449411:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760792.449420:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534091776 00000100:00100000:0.0:1789760792.449425:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534091776 00000100:00100000:0.0:1789760792.449451:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 996 00000100:00100000:0.0:1789760792.449459:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3475:x1876699534091776:12345-0@lo:400:kworker.0 00002000:00000024:3.0:1789760792.449690:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3832545280 avail 3796893696 00000100:00100000:0.0:1789760792.449727:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.449763:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04984e7f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534092672:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.449831:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3473:x1876699534093056:12345-0@lo:13:osp-pre-0-0.0 Request processed in 716us (1701us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760792.449889:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:3.0:1789760792.449971:0:3473:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888131c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3473:1876699534093056:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760792.449995:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3475:x1876699534091776:12345-0@lo:400:kworker.0 Request processed in 534us (1259us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760792.450004:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 996 00000100:00100000:1.0:1789760792.450021:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534092160 00000100:00100000:1.0:1789760792.450025:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534092160 00000100:00080000:3.0:1789760792.450324:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760792.450555:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760792.450636:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760792.450670:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04987254380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534092416:0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.450720:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760792.450746:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04984e7f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534092928:0@lo:400:kworker.0 00000100:00100000:1.0:1789760792.450782:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534092672 00000100:00100000:1.0:1789760792.450787:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534092672 00000100:00100000:1.0:1789760792.450820:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 997 00000100:00100000:3.0:1789760792.450828:0:8128:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 998 00000100:00100000:1.0:1789760792.450831:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534092160:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.450926:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7340380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3474:1876699534091648:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.450956:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534092416 00000100:00100000:0.0:1789760792.450960:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534092416 00000100:00100000:0.0:1789760792.450988:0:8102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 254 00000100:00100000:0.0:1789760792.450995:0:8102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534092416:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.451029:0:8128:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534092672:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.451595:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760792.451650:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0499b66df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3472:1876699534091392:0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.451781:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534091904 00000100:00100000:2.0:1789760792.451787:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534091904 00000100:00100000:2.0:1789760792.451824:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:2.0:1789760792.451833:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534091904:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.451886:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534091904:12345-0@lo:400:kworker.0 Request processed in 54us (1251us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760792.451893:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:1.0:1789760792.452938:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3474:x1876699534092160:12345-0@lo:400:kworker.0 Request processed in 2090us (3193us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760792.452948:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 997 00000100:00100000:1.0:1789760792.453028:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04984e7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534093184:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760792.453117:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048851d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534093312:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760792.453176:0:8102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534092416:12345-0@lo:400:kworker.0 Request processed in 2178us (2453us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.453188:0:8102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 254 00000100:00100000:0.0:1789760792.453217:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534092928 00000100:00100000:0.0:1789760792.453223:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534092928 00000100:00100000:1.0:1789760792.453246:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7340e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3475:1876699534091776:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.453263:0:8102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 255 00000100:00100000:1.0:1789760792.453270:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04987255180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534092288:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.453277:0:8102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534092928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760792.453307:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04984e7ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534092800:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.453346:0:8102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534092928:12345-0@lo:400:kworker.0 Request processed in 74us (1830us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.453355:0:8102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 255 00000100:00100000:0.0:1789760792.453486:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04987255880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3474:1876699534092160:0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.453723:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888046a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534093440:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760792.453817:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760792.454665:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049b7342680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534091904:0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.454682:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04987254380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534092416:0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.454706:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04984e7f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534092928:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.455597:0:8128:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534092672:12345-0@lo:400:kworker.0 Request processed in 4752us (5034us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760792.455611:0:8128:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 998 00000100:00080000:2.0:1789760792.455636:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760792.455706:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04984e7f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534092672:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.455904:0:9367:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534093440 00000100:00100000:0.0:1789760792.455912:0:9367:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534093440 00000100:00100000:0.0:1789760792.455970:0:9367:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00100000:0.0:1789760792.455980:0:9367:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534093440:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760792.456011:0:9367:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3832545280 avail 3796893696 00000100:00100000:0.0:1789760792.456069:0:9367:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534093440:12345-0@lo:13:osp-pre-0-1.0 Request processed in 90us (2252us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.456077:0:9367:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00000100:00100000:0.0:1789760792.456125:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888046a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534093440:0@lo:13:osp-pre-0-1.0 00000001:00080000:1.0:1789760792.495905:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973538 is committed 00002000:00010000:0.0:1789760792.496013:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25468928 -> 25473024 00010000:00010000:0.0:1789760792.496034:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe57115 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.496336:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe57115 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.496347:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe57115 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.496354:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760792.496360:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe57115 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.496372:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe57115 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.496397:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe57115 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.496406:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe57115 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25468928->25473023] (req 25468928->25473023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.496569:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508483584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 81357us (82848us total) trans 4294973538 rc 0/0 00000100:00100000:0.0:1789760792.496582:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6250 00000100:00100000:0.0:1789760792.496653:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771441934931 time 771519898043 state 0x2 ref 0 00000100:00100000:0.0:1789760792.499024:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760792.499216:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508483712 00000100:00100000:3.0:1789760792.499229:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508483712 00000100:00100000:3.0:1789760792.499327:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771441934931 time 771519898043 state 0x0 ref 1 00000100:00100000:3.0:1789760792.499346:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771520158459 time 771519898043 state 0x1 ref 1 00000100:00100000:3.0:1789760792.499377:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771520158459 time 771604087471 state 0x0 ref 1 00000100:00100000:3.0:1789760792.499390:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508483712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760792.499453:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25473024, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0xc807 00010000:00010000:3.0:1789760792.499463:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5711c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.499482:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.499487:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.499495:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5711c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.499512:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5711c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760792.540278:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25473024 -> 25477120 00002000:00010000:3.0:1789760792.540311:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760792 -> 1789760793 00002000:00010000:3.0:1789760792.540314:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760792 -> 1789760793 00010000:00010000:3.0:1789760792.540332:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5711c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.540348:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5711c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.540359:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5711c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.540368:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760792.540374:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5711c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.540385:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5711c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.540410:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5711c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.540425:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe5711c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25473024->25477119] (req 25473024->25477119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.540539:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508483712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41142us (41522us total) trans 4294973539 rc 0/0 00000100:00100000:3.0:1789760792.540551:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6251 00000100:00100000:3.0:1789760792.540563:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771520158459 time 771604087471 state 0x2 ref 0 00000001:00080000:1.0:1789760792.542705:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973539 is committed 00000100:00100000:2.0:1789760792.543664:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760792.543840:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508483840 00000100:00100000:3.0:1789760792.543853:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508483840 00000100:00100000:3.0:1789760792.543925:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771520158459 time 771604087471 state 0x0 ref 1 00000100:00100000:3.0:1789760792.543944:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771604347887 time 771604087471 state 0x1 ref 1 00000100:00100000:3.0:1789760792.543973:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771604347887 time 771648683937 state 0x0 ref 1 00000100:00100000:3.0:1789760792.543988:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508483840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760792.544063:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25477120, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:3.0:1789760792.544073:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57123 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.544093:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.544099:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.544106:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57123 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.544129:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57123 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760792.573832:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25477120 -> 25481216 00010000:00010000:3.0:1789760792.573861:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57123 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.573877:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57123 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.573886:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57123 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.573894:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760792.573900:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57123 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.573910:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57123 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.573938:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57123 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.573946:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57123 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25477120->25481215] (req 25477120->25481215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.574198:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508483840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30205us (30535us total) trans 4294973540 rc 0/0 00000100:00100000:3.0:1789760792.574211:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6252 00000100:00100000:3.0:1789760792.574224:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771604347887 time 771648683937 state 0x2 ref 0 00000001:00080000:2.0:1789760792.574330:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973540 is committed 00000100:00100000:2.0:1789760792.579850:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760792.579962:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508483968 00000100:00100000:0.0:1789760792.579979:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508483968 00000100:00100000:0.0:1789760792.580122:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771604347887 time 771648683937 state 0x0 ref 1 00000100:00100000:0.0:1789760792.580142:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771648944353 time 771648683937 state 0x1 ref 1 00000100:00100000:0.0:1789760792.580207:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771648944353 time 771684917332 state 0x0 ref 1 00000100:00100000:0.0:1789760792.580222:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508483968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760792.580308:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25481216, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:0.0:1789760792.580320:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5712a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.580339:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:0.0:1789760792.580344:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.580351:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5712a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.580368:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5712a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.625934:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973541 is committed 00002000:00010000:0.0:1789760792.626025:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25481216 -> 25485312 00010000:00010000:0.0:1789760792.626056:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5712a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.626071:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5712a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.626081:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5712a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.626090:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760792.626095:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5712a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.626106:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5712a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.626133:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5712a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.626142:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe5712a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25481216->25485311] (req 25481216->25485311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.626256:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508483968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46030us (46415us total) trans 4294973541 rc 0/0 00000100:00100000:0.0:1789760792.626268:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6253 00000100:00100000:0.0:1789760792.626280:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771648944353 time 771684917332 state 0x2 ref 0 00000100:00100000:1.0:1789760792.632699:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760792.632966:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508484096 00000100:00100000:3.0:1789760792.632981:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508484096 00000100:00100000:3.0:1789760792.633813:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771648944353 time 771684917332 state 0x0 ref 1 00000100:00100000:3.0:1789760792.633835:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771685177748 time 771684917332 state 0x1 ref 1 00000100:00100000:3.0:1789760792.633879:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771685177748 time 771738588337 state 0x0 ref 1 00000100:00100000:3.0:1789760792.633897:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508484096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760792.633963:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25485312, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:3.0:1789760792.633975:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57131 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.633994:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:3.0:1789760792.634010:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.634018:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57131 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.634036:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57131 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.703724:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973542 is committed 00002000:00010000:3.0:1789760792.703858:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25485312 -> 25489408 00010000:00010000:3.0:1789760792.703877:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57131 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.703894:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57131 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.703903:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57131 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.703910:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760792.703916:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57131 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.703926:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57131 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.703951:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57131 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.703959:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57131 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25485312->25489407] (req 25485312->25489407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.704098:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888130700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508484096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 70201us (71415us total) trans 4294973542 rc 0/0 00000100:00100000:3.0:1789760792.704114:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6254 00000100:00100000:3.0:1789760792.704126:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771685177748 time 771738588337 state 0x2 ref 0 00000100:00100000:2.0:1789760792.710370:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760792.710638:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508484224 00000100:00100000:1.0:1789760792.710652:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508484224 00000100:00100000:1.0:1789760792.710728:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771685177748 time 771738588337 state 0x0 ref 1 00000100:00100000:1.0:1789760792.710747:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771738848753 time 771738588337 state 0x1 ref 1 00000100:00100000:1.0:1789760792.710776:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771738848753 time 771815486164 state 0x0 ref 1 00000100:00100000:1.0:1789760792.710789:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508484224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760792.710851:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25489408, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:1.0:1789760792.710863:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57138 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.710882:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760792.710887:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.710894:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57138 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.710911:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57138 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.751233:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973543 is committed 00002000:00010000:1.0:1789760792.751350:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25489408 -> 25493504 00010000:00010000:1.0:1789760792.751372:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57138 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.751387:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57138 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.751397:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57138 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.751404:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760792.751409:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57138 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.751424:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57138 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.751450:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57138 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.751462:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57138 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25489408->25493503] (req 25489408->25493503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.751589:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508484224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40794us (41226us total) trans 4294973543 rc 0/0 00000100:00100000:1.0:1789760792.751605:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6255 00000100:00100000:1.0:1789760792.751617:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771738848753 time 771815486164 state 0x2 ref 0 00000100:00100000:0.0:1789760792.754528:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760792.755024:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508484352 00000100:00100000:2.0:1789760792.755036:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508484352 00000100:00100000:2.0:1789760792.755149:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771738848753 time 771815486164 state 0x0 ref 1 00000100:00100000:2.0:1789760792.755164:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771815746580 time 771815486164 state 0x1 ref 1 00000100:00100000:2.0:1789760792.755194:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771815746580 time 771859904017 state 0x0 ref 1 00000100:00100000:2.0:1789760792.755208:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508484352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760792.755374:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25493504, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:2.0:1789760792.755386:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5713f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.755418:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760792.755424:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.755432:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5713f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.755578:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5713f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.811727:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973544 is committed 00002000:00010000:1.0:1789760792.811995:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25493504 -> 25497600 00010000:00010000:1.0:1789760792.812015:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5713f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.812030:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5713f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.812101:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5713f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.812111:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760792.812117:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5713f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.812128:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5713f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.812154:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5713f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.812163:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe5713f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25493504->25497599] (req 25493504->25497599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.812383:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508484352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57171us (57863us total) trans 4294973544 rc 0/0 00000100:00100000:1.0:1789760792.812395:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6256 00000100:00100000:1.0:1789760792.812407:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771815746580 time 771859904017 state 0x2 ref 0 00000100:00100000:3.0:1789760792.819583:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760792.824648:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508484480 00000100:00100000:2.0:1789760792.824668:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508484480 00000100:00100000:2.0:1789760792.824777:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771815746580 time 771859904017 state 0x0 ref 1 00000100:00100000:2.0:1789760792.824970:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771860164433 time 771859904017 state 0x1 ref 1 00000100:00100000:2.0:1789760792.825014:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771860164433 time 771929724137 state 0x0 ref 1 00000100:00100000:2.0:1789760792.825031:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508484480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760792.825160:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25497600, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:2.0:1789760792.825172:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57146 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.825191:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:2.0:1789760792.825196:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.825318:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57146 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.825336:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57146 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760792.858261:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25497600 -> 25501696 00010000:00010000:1.0:1789760792.858308:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57146 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.858325:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57146 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.858336:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57146 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.858345:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760792.858357:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57146 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.858369:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57146 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.858385:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973545 is committed 00010000:00010000:1.0:1789760792.858392:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57146 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.858405:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57146 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25497600->25501695] (req 25497600->25501695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.858515:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508484480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33482us (38936us total) trans 4294973545 rc 0/0 00000100:00100000:1.0:1789760792.858528:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6257 00000100:00100000:1.0:1789760792.858540:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771860164433 time 771929724137 state 0x2 ref 0 00000100:00100000:0.0:1789760792.862817:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760792.862930:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508484608 00000100:00100000:1.0:1789760792.862944:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508484608 00000100:00100000:1.0:1789760792.863022:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771860164433 time 771929724137 state 0x0 ref 1 00000100:00100000:1.0:1789760792.863042:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771929984553 time 771929724137 state 0x1 ref 1 00000100:00100000:1.0:1789760792.863081:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771929984553 time 771967790702 state 0x0 ref 1 00000100:00100000:1.0:1789760792.863097:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508484608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760792.863163:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25501696, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:1.0:1789760792.863174:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5714d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.863194:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=771s v=(0 0) 00010000:00010000:1.0:1789760792.863200:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.863207:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5714d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.863231:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5714d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.901762:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973546 is committed 00002000:00010000:2.0:1789760792.901882:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25501696 -> 25505792 00010000:00010000:2.0:1789760792.901902:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5714d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.901916:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5714d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.901925:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5714d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.901937:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760792.901943:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5714d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.901952:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5714d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.901977:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5714d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.901984:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe5714d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25501696->25505791] (req 25501696->25505791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.902114:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508484608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39013us (39304us total) trans 4294973546 rc 0/0 00000100:00100000:2.0:1789760792.902127:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6258 00000100:00100000:2.0:1789760792.902139:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771929984553 time 771967790702 state 0x2 ref 0 00000100:00100000:0.0:1789760792.910157:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760792.910336:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508484736 00000100:00100000:1.0:1789760792.910348:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508484736 00000100:00100000:1.0:1789760792.910705:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771929984553 time 771967790702 state 0x0 ref 1 00000100:00100000:1.0:1789760792.910726:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771968051118 time 771967790702 state 0x1 ref 1 00000100:00100000:1.0:1789760792.910861:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771968051118 time 772015569509 state 0x0 ref 1 00000100:00100000:1.0:1789760792.910880:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508484736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760792.910951:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25505792, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:1.0:1789760792.910963:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57154 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.910984:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760792.910990:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.910998:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57154 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.911017:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57154 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.953066:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973547 is committed 00002000:00010000:3.0:1789760792.953226:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25505792 -> 25509888 00010000:00010000:3.0:1789760792.953246:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57154 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.953262:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57154 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.953272:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57154 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.953281:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760792.953306:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57154 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.953318:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57154 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.953344:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57154 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.953353:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57154 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25505792->25509887] (req 25505792->25509887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.953474:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508484736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42592us (43445us total) trans 4294973547 rc 0/0 00000100:00100000:3.0:1789760792.953487:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6259 00000100:00100000:3.0:1789760792.953500:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771968051118 time 772015569509 state 0x2 ref 0 00000100:00100000:1.0:1789760792.956421:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760792.956523:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508484864 00000100:00100000:2.0:1789760792.956537:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508484864 00000100:00100000:2.0:1789760792.956669:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 771968051118 time 772015569509 state 0x0 ref 1 00000100:00100000:2.0:1789760792.956699:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772015829925 time 772015569509 state 0x1 ref 1 00000100:00100000:2.0:1789760792.956766:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772015829925 time 772061475601 state 0x0 ref 1 00000100:00100000:2.0:1789760792.956783:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508484864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760792.956848:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25509888, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:2.0:1789760792.956859:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5715b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.956880:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760792.956885:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.956892:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5715b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.956908:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5715b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.992879:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973548 is committed 00002000:00010000:1.0:1789760792.993009:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25509888 -> 25513984 00010000:00010000:1.0:1789760792.993031:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5715b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.993057:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5715b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.993067:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5715b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.993075:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760792.993081:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5715b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.993092:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5715b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.993117:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5715b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.993126:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5715b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25509888->25513983] (req 25509888->25513983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.993257:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888133800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508484864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36469us (36841us total) trans 4294973548 rc 0/0 00000100:00100000:1.0:1789760792.993271:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6260 00000100:00100000:1.0:1789760792.993551:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772015829925 time 772061475601 state 0x2 ref 0 00000100:00100000:0.0:1789760792.997120:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760792.997234:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508484992 00000100:00100000:2.0:1789760792.997248:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508484992 00000100:00100000:2.0:1789760792.997386:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772015829925 time 772061475601 state 0x0 ref 1 00000100:00100000:2.0:1789760792.997406:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772061736017 time 772061475601 state 0x1 ref 1 00000100:00100000:2.0:1789760792.997476:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772061736017 time 772102185611 state 0x0 ref 1 00000100:00100000:2.0:1789760792.997492:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508484992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760792.997555:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25513984, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:2.0:1789760792.997568:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57162 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.997595:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760792.997608:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.997617:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57162 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.997634:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57162 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.033482:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973549 is committed 00002000:00010000:2.0:1789760793.033609:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25513984 -> 25518080 00010000:00010000:2.0:1789760793.033629:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57162 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.033645:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57162 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.033655:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57162 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.033663:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760793.033668:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57162 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.033679:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57162 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.033712:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57162 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.033721:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57162 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25513984->25518079] (req 25513984->25518079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.033883:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508484992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36388us (36771us total) trans 4294973549 rc 0/0 00000100:00100000:2.0:1789760793.033895:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6261 00000100:00100000:2.0:1789760793.033907:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772061736017 time 772102185611 state 0x2 ref 0 00000100:00100000:0.0:1789760793.036937:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760793.037100:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508485120 00000100:00100000:3.0:1789760793.037115:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508485120 00000100:00100000:3.0:1789760793.037246:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772061736017 time 772102185611 state 0x0 ref 1 00000100:00100000:3.0:1789760793.037265:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772102446027 time 772102185611 state 0x1 ref 1 00000100:00100000:3.0:1789760793.037318:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772102446027 time 772142027964 state 0x0 ref 1 00000100:00100000:3.0:1789760793.037335:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508485120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760793.037397:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25518080, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:3.0:1789760793.037408:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57169 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.037429:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.037434:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.037442:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57169 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.037459:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57169 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760793.038268:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760793.070928:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973550 is committed 00002000:00010000:0.0:1789760793.071137:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25518080 -> 25522176 00010000:00010000:0.0:1789760793.071158:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57169 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.071179:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57169 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.071190:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57169 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.071199:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760793.071204:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57169 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.071215:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57169 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.071240:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57169 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.071249:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57169 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25518080->25522175] (req 25518080->25522175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.071732:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508485120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34394us (34801us total) trans 4294973550 rc 0/0 00000100:00100000:0.0:1789760793.071750:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6262 00000100:00100000:0.0:1789760793.071763:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772102446027 time 772142027964 state 0x2 ref 0 00000100:00100000:1.0:1789760793.075164:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760793.075459:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508485248 00000100:00100000:2.0:1789760793.075475:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508485248 00000100:00100000:3.0:1789760793.075702:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772102446027 time 772142027964 state 0x0 ref 1 00000100:00100000:3.0:1789760793.075722:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772142288380 time 772142027964 state 0x1 ref 1 00000100:00100000:3.0:1789760793.075794:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772142288380 time 772180502721 state 0x0 ref 1 00000100:00100000:3.0:1789760793.075809:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508485248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760793.075869:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25522176, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:3.0:1789760793.075880:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57170 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.075901:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.075907:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.075915:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57170 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.075933:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57170 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.107498:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973551 is committed 00002000:00010000:1.0:1789760793.107564:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25522176 -> 25526272 00010000:00010000:1.0:1789760793.107583:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57170 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.107598:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57170 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.107673:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57170 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.107681:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760793.107696:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57170 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.107707:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57170 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.107734:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57170 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.107742:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57170 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25522176->25526271] (req 25522176->25526271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.107860:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508485248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32047us (32918us total) trans 4294973551 rc 0/0 00000100:00100000:1.0:1789760793.107873:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6263 00000100:00100000:1.0:1789760793.107887:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772142288380 time 772180502721 state 0x2 ref 0 00000100:00100000:1.0:1789760793.109964:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760793.110029:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508485376 00000100:00100000:2.0:1789760793.110040:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508485376 00000100:00100000:2.0:1789760793.110117:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772142288380 time 772180502721 state 0x0 ref 1 00000100:00100000:2.0:1789760793.110136:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772180763137 time 772180502721 state 0x1 ref 1 00000100:00100000:2.0:1789760793.110171:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772180763137 time 772214881225 state 0x0 ref 1 00000100:00100000:2.0:1789760793.110182:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888132a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508485376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760793.110247:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25526272, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:2.0:1789760793.110259:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57177 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.110315:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760793.110321:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.110330:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57177 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.110347:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57177 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.165474:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973552 is committed 00002000:00010000:1.0:1789760793.165713:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25526272 -> 25530368 00010000:00010000:1.0:1789760793.165734:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57177 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.165749:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57177 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.165760:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57177 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.165768:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760793.165775:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57177 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.165785:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57177 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.165811:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57177 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.165819:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012495cd0/0xb49177c0efe57177 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25526272->25530367] (req 25526272->25530367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.165948:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888132a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508485376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55759us (55989us total) trans 4294973552 rc 0/0 00000100:00100000:1.0:1789760793.165960:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6264 00000100:00100000:1.0:1789760793.165972:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772180763137 time 772214881225 state 0x2 ref 0 00000100:00100000:0.0:1789760793.169894:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760793.170136:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508485504 00000100:00100000:3.0:1789760793.170151:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508485504 00000100:00100000:3.0:1789760793.170226:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772180763137 time 772214881225 state 0x0 ref 1 00000100:00100000:3.0:1789760793.170247:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772215141641 time 772214881225 state 0x1 ref 1 00000100:00100000:3.0:1789760793.170283:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772215141641 time 772274992917 state 0x0 ref 1 00000100:00100000:3.0:1789760793.170323:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508485504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760793.170393:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25530368, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:3.0:1789760793.170405:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5717e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.170427:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.170432:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.170440:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5717e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.170459:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5717e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.261492:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25530368 -> 25534464 00010000:00010000:3.0:1789760793.261521:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5717e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.261537:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5717e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.261547:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5717e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.261555:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760793.261562:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5717e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.261571:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5717e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.261596:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5717e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.261605:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5717e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25530368->25534463] (req 25530368->25534463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.261756:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508485504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 91429us (91871us total) trans 4294973553 rc 0/0 00000100:00100000:3.0:1789760793.261770:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6265 00000100:00100000:3.0:1789760793.261782:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772215141641 time 772274992917 state 0x2 ref 0 00000001:00080000:1.0:1789760793.262507:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973553 is committed 00000100:00100000:3.0:1789760793.265493:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760793.266028:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508485632 00000100:00100000:1.0:1789760793.266051:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508485632 00000100:00100000:1.0:1789760793.266127:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772215141641 time 772274992917 state 0x0 ref 1 00000100:00100000:1.0:1789760793.266146:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772275253333 time 772274992917 state 0x1 ref 1 00000100:00100000:1.0:1789760793.266184:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772275253333 time 772370894198 state 0x0 ref 1 00000100:00100000:1.0:1789760793.266197:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508485632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760793.266263:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25534464, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:1.0:1789760793.266274:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57185 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.266339:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.266345:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.266352:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57185 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.266370:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57185 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.326104:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25534464 -> 25538560 00010000:00010000:1.0:1789760793.326132:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57185 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.326148:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57185 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.326158:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57185 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.326167:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760793.326172:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57185 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.326184:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57185 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.326209:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57185 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.326218:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57185 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25534464->25538559] (req 25534464->25538559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.326401:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508485632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60198us (60911us total) trans 4294973554 rc 0/0 00000001:00080000:0.0:1789760793.326403:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973554 is committed 00000100:00100000:1.0:1789760793.326426:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6266 00000100:00100000:1.0:1789760793.326440:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772275253333 time 772370894198 state 0x2 ref 0 00000100:00100000:2.0:1789760793.331077:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760793.331217:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508485760 00000100:00100000:1.0:1789760793.331232:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508485760 00000100:00100000:1.0:1789760793.331347:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772275253333 time 772370894198 state 0x0 ref 1 00000100:00100000:1.0:1789760793.331368:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772371154614 time 772370894198 state 0x1 ref 1 00000100:00100000:1.0:1789760793.331402:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772371154614 time 772436111415 state 0x0 ref 1 00000100:00100000:1.0:1789760793.331416:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508485760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760793.331573:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25538560, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:1.0:1789760793.331584:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5718c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.331614:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.331619:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.331627:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5718c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.331644:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5718c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.342616:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760793.379015:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25538560 -> 25542656 00010000:00010000:1.0:1789760793.379175:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5718c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.379193:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5718c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.379204:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5718c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.379213:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760793.379218:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5718c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.379229:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5718c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.379342:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5718c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.379352:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5718c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25538560->25542655] (req 25538560->25542655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.379535:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508485760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48113us (48465us total) trans 4294973555 rc 0/0 00000100:00100000:1.0:1789760793.379549:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6267 00000100:00100000:1.0:1789760793.379563:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772371154614 time 772436111415 state 0x2 ref 0 00000001:00080000:1.0:1789760793.381161:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973555 is committed 00000100:00100000:1.0:1789760793.387075:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760793.387252:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508485888 00000100:00100000:3.0:1789760793.387267:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508485888 00000100:00100000:3.0:1789760793.387375:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772371154614 time 772436111415 state 0x0 ref 1 00000100:00100000:3.0:1789760793.387395:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772436371831 time 772436111415 state 0x1 ref 1 00000100:00100000:3.0:1789760793.387434:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772436371831 time 772492143681 state 0x0 ref 1 00000100:00100000:3.0:1789760793.387450:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508485888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760793.387727:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25542656, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:3.0:1789760793.387740:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57193 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.387761:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.387766:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.387774:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57193 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.387791:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57193 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.429308:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973556 is committed 00002000:00010000:1.0:1789760793.429489:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25542656 -> 25546752 00010000:00010000:1.0:1789760793.429510:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57193 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.429526:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57193 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.429536:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57193 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.429544:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760793.429550:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57193 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.429560:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57193 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.429585:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57193 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.429594:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57193 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25542656->25546751] (req 25542656->25546751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.429722:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508485888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 42269us (42659us total) trans 4294973556 rc 0/0 00000100:00100000:1.0:1789760793.429735:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6268 00000100:00100000:1.0:1789760793.429748:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772436371831 time 772492143681 state 0x2 ref 0 00000100:00100000:0.0:1789760793.432790:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760793.432916:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508486016 00000100:00100000:1.0:1789760793.432928:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508486016 00000100:00100000:1.0:1789760793.433003:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772436371831 time 772492143681 state 0x0 ref 1 00000100:00100000:1.0:1789760793.433022:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772492404097 time 772492143681 state 0x1 ref 1 00000100:00100000:1.0:1789760793.433092:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772492404097 time 772537801340 state 0x0 ref 1 00000100:00100000:1.0:1789760793.433106:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508486016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760793.433171:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25546752, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:1.0:1789760793.433182:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5719a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.433201:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.433206:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.433214:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5719a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.433231:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5719a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.472828:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973557 is committed 00002000:00010000:1.0:1789760793.473014:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25546752 -> 25550848 00010000:00010000:1.0:1789760793.473041:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5719a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.473061:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5719a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.473071:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5719a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.473080:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760793.473085:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5719a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.473096:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5719a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.473121:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5719a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.473128:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5719a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25546752->25550847] (req 25546752->25550847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.473242:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508486016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40133us (40458us total) trans 4294973557 rc 0/0 00000100:00100000:1.0:1789760793.473253:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6269 00000100:00100000:1.0:1789760793.473265:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772492404097 time 772537801340 state 0x2 ref 0 00000100:00100000:0.0:1789760793.474353:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760793.474379:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04984e7d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534093568:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760793.474446:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760793.474560:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534093568 00000100:00100000:1.0:1789760793.474571:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534093568 00000100:00100000:1.0:1789760793.474626:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00100000:1.0:1789760793.474637:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534093568:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760793.474703:0:6471:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760793.474753:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534093568:12345-0@lo:41:osp-pre-0-1.0 Request processed in 117us (307us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760793.474760:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00100000:0.0:1789760793.475359:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04984e7d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534093568:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1789760793.480390:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760793.480533:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508486144 00000100:00100000:1.0:1789760793.480547:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508486144 00000100:00100000:1.0:1789760793.480625:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772492404097 time 772537801340 state 0x0 ref 1 00000100:00100000:1.0:1789760793.480645:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772538061756 time 772537801340 state 0x1 ref 1 00000100:00100000:1.0:1789760793.480923:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772538061756 time 772585632409 state 0x0 ref 1 00000100:00100000:1.0:1789760793.480940:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508486144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760793.481005:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25550848, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0xc807 00010000:00010000:1.0:1789760793.481015:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe571a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.481045:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.481052:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.481059:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe571a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.481075:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe571a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.506923:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973558 is committed 00002000:00010000:3.0:1789760793.506980:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25550848 -> 25554944 00002000:00010000:3.0:1789760793.506986:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760793 -> 1789760794 00002000:00010000:3.0:1789760793.506989:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760793 -> 1789760794 00010000:00010000:3.0:1789760793.507007:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe571a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.507023:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe571a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.507033:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe571a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.507042:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760793.507048:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe571a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.507060:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe571a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.507086:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe571a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.507095:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe571a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25550848->25554943] (req 25550848->25554943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.507218:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508486144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26274us (26836us total) trans 4294973558 rc 0/0 00000100:00100000:3.0:1789760793.507231:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6270 00000100:00100000:3.0:1789760793.507243:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772538061756 time 772585632409 state 0x2 ref 0 00000100:00100000:0.0:1789760793.516062:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760793.516229:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508486272 00000100:00100000:1.0:1789760793.516241:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508486272 00000100:00100000:1.0:1789760793.516345:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772538061756 time 772585632409 state 0x0 ref 1 00000100:00100000:1.0:1789760793.516364:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772585892825 time 772585632409 state 0x1 ref 1 00000100:00100000:1.0:1789760793.516391:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772585892825 time 772621100711 state 0x0 ref 1 00000100:00100000:1.0:1789760793.516405:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508486272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760793.516472:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25554944, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:1.0:1789760793.516482:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe571a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.516501:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.516506:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.516513:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe571a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.516529:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe571a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.549231:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973559 is committed 00002000:00010000:3.0:1789760793.549467:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25554944 -> 25559040 00010000:00010000:3.0:1789760793.549489:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe571a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.549504:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe571a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.549514:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe571a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.549521:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760793.549528:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe571a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.549538:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe571a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.549563:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe571a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.549572:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe571a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25554944->25559039] (req 25554944->25559039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.549749:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987254380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508486272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33338us (33692us total) trans 4294973559 rc 0/0 00000100:00100000:3.0:1789760793.549760:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6271 00000100:00100000:3.0:1789760793.549772:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772585892825 time 772621100711 state 0x2 ref 0 00000100:00100000:0.0:1789760793.553593:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760793.553707:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508486400 00000100:00100000:1.0:1789760793.553719:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508486400 00000100:00100000:1.0:1789760793.553792:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772585892825 time 772621100711 state 0x0 ref 1 00000100:00100000:1.0:1789760793.553810:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772621361127 time 772621100711 state 0x1 ref 1 00000100:00100000:1.0:1789760793.553892:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772621361127 time 772658601690 state 0x0 ref 1 00000100:00100000:1.0:1789760793.553905:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508486400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760793.553967:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25559040, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:1.0:1789760793.553978:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe571af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.553997:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.554001:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.554009:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe571af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.554026:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe571af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.581795:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973560 is committed 00002000:00010000:0.0:1789760793.581967:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25559040 -> 25563136 00010000:00010000:0.0:1789760793.581988:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe571af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.582004:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe571af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.582014:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe571af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.582023:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760793.582028:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe571af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.582038:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe571af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.582066:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe571af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.582074:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe571af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25559040->25563135] (req 25559040->25563135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.582211:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508486400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28299us (28694us total) trans 4294973560 rc 0/0 00000100:00100000:0.0:1789760793.582222:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6272 00000100:00100000:0.0:1789760793.582235:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772621361127 time 772658601690 state 0x2 ref 0 00000100:00100000:2.0:1789760793.589586:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760793.589692:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508486528 00000100:00100000:3.0:1789760793.589705:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508486528 00000100:00100000:3.0:1789760793.589778:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772621361127 time 772658601690 state 0x0 ref 1 00000100:00100000:3.0:1789760793.589797:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772658862106 time 772658601690 state 0x1 ref 1 00000100:00100000:3.0:1789760793.589833:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772658862106 time 772694543383 state 0x0 ref 1 00000100:00100000:3.0:1789760793.589846:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508486528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760793.589913:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25563136, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:3.0:1789760793.589924:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.589943:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:3.0:1789760793.589948:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.589954:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.589971:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.626902:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973561 is committed 00002000:00010000:3.0:1789760793.626998:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25563136 -> 25567232 00010000:00010000:3.0:1789760793.627019:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.627154:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.627166:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.627174:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760793.627180:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.627190:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.627217:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.627226:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25563136->25567231] (req 25563136->25567231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.627480:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508486528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37627us (37900us total) trans 4294973561 rc 0/0 00000100:00100000:3.0:1789760793.627494:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6273 00000100:00100000:3.0:1789760793.627507:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772658862106 time 772694543383 state 0x2 ref 0 00000100:00100000:3.0:1789760793.633011:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760793.633639:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508486656 00000100:00100000:1.0:1789760793.633654:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508486656 00000100:00100000:1.0:1789760793.634018:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772658862106 time 772694543383 state 0x0 ref 1 00000100:00100000:1.0:1789760793.634039:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772694803799 time 772694543383 state 0x1 ref 1 00000100:00100000:1.0:1789760793.634140:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772694803799 time 772738848604 state 0x0 ref 1 00000100:00100000:1.0:1789760793.634156:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508486656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760793.634233:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25567232, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:1.0:1789760793.634245:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.634264:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.634270:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.634277:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.634355:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760793.680970:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25567232 -> 25571328 00010000:00010000:1.0:1789760793.681001:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.681017:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.681028:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.681046:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760793.681052:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.681062:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.681088:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.681097:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25567232->25571327] (req 25567232->25571327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.681226:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508486656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47064us (48270us total) trans 4294973562 rc 0/0 00000100:00100000:1.0:1789760793.681238:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6274 00000100:00100000:1.0:1789760793.681250:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772694803799 time 772738848604 state 0x2 ref 0 00000001:00080000:2.0:1789760793.681395:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973562 is committed 00000100:00100000:3.0:1789760793.689259:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760793.693469:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508486784 00000100:00100000:0.0:1789760793.693495:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508486784 00000100:00100000:2.0:1789760793.695040:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772694803799 time 772738848604 state 0x0 ref 1 00000100:00100000:2.0:1789760793.695064:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772739109020 time 772738848604 state 0x1 ref 1 00000100:00100000:2.0:1789760793.695118:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772739109020 time 772799827621 state 0x0 ref 1 00000100:00100000:2.0:1789760793.695143:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508486784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760793.695214:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25571328, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:2.0:1789760793.695226:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe571c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.695247:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760793.695253:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.695261:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe571c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.695279:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe571c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.733498:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973563 is committed 00002000:00010000:2.0:1789760793.733626:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25571328 -> 25575424 00010000:00010000:2.0:1789760793.733647:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe571c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.733662:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe571c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.733672:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe571c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.733680:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760793.733697:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe571c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.733708:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe571c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.733733:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe571c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.733746:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe571c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25571328->25575423] (req 25571328->25575423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.733868:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508486784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38730us (44616us total) trans 4294973563 rc 0/0 00000100:00100000:2.0:1789760793.733880:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6275 00000100:00100000:2.0:1789760793.733892:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772739109020 time 772799827621 state 0x2 ref 0 00000100:00100000:3.0:1789760793.736124:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760793.736305:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508486912 00000100:00100000:1.0:1789760793.736318:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508486912 00000100:00100000:1.0:1789760793.736382:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772739109020 time 772799827621 state 0x0 ref 1 00000100:00100000:1.0:1789760793.736400:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772800088037 time 772799827621 state 0x1 ref 1 00000100:00100000:1.0:1789760793.736431:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772800088037 time 772841141401 state 0x0 ref 1 00000100:00100000:1.0:1789760793.736443:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508486912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760793.736504:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25575424, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:1.0:1789760793.736515:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.736535:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.736540:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.736548:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.736564:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.774661:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973564 is committed 00002000:00010000:1.0:1789760793.774942:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25575424 -> 25579520 00010000:00010000:1.0:1789760793.774963:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.774988:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.774999:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.775007:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760793.775012:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.775022:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.775056:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.775065:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25575424->25579519] (req 25575424->25579519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.775188:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508486912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38739us (39158us total) trans 4294973564 rc 0/0 00000100:00100000:1.0:1789760793.775200:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6276 00000100:00100000:1.0:1789760793.775213:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772800088037 time 772841141401 state 0x2 ref 0 00000100:00100000:3.0:1789760793.781425:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760793.781518:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508487040 00000100:00100000:1.0:1789760793.781532:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508487040 00000100:00100000:1.0:1789760793.781611:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772800088037 time 772841141401 state 0x0 ref 1 00000100:00100000:1.0:1789760793.781630:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772841401817 time 772841141401 state 0x1 ref 1 00000100:00100000:1.0:1789760793.781752:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772841401817 time 772886460517 state 0x0 ref 1 00000100:00100000:1.0:1789760793.781768:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508487040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760793.781838:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25579520, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:1.0:1789760793.781849:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.781869:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.781874:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.781881:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.781897:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.838437:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973565 is committed 00002000:00010000:3.0:1789760793.838548:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25579520 -> 25583616 00010000:00010000:3.0:1789760793.838565:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.838576:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.838583:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.838592:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760793.838650:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.838658:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.838679:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.838697:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe571d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25579520->25583615] (req 25579520->25583615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.838815:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508487040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57044us (57396us total) trans 4294973565 rc 0/0 00000100:00100000:3.0:1789760793.838826:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6277 00000100:00100000:3.0:1789760793.838838:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772841401817 time 772886460517 state 0x2 ref 0 00000100:00100000:3.0:1789760793.841548:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760793.841631:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508487168 00000100:00100000:2.0:1789760793.841644:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508487168 00000100:00100000:2.0:1789760793.841726:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772841401817 time 772886460517 state 0x0 ref 1 00000100:00100000:2.0:1789760793.841744:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772886720933 time 772886460517 state 0x1 ref 1 00000100:00100000:2.0:1789760793.841785:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772886720933 time 772946495409 state 0x0 ref 1 00000100:00100000:2.0:1789760793.841799:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508487168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760793.841863:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25583616, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:2.0:1789760793.841874:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe571d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.841894:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:2.0:1789760793.841898:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.841905:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe571d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.841918:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe571d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760793.872409:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25583616 -> 25587712 00010000:00010000:3.0:1789760793.872437:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe571d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.872450:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe571d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.872462:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe571d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.872468:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760793.872472:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe571d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.872479:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe571d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.872498:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe571d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.872504:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe571d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25583616->25587711] (req 25583616->25587711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.872748:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973566 is committed 00000100:00100000:3.0:1789760793.872824:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508487168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31020us (31283us total) trans 4294973566 rc 0/0 00000100:00100000:3.0:1789760793.872835:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6278 00000100:00100000:3.0:1789760793.872847:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772886720933 time 772946495409 state 0x2 ref 0 00000100:00100000:2.0:1789760793.881426:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760793.881645:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508487296 00000100:00100000:1.0:1789760793.881660:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508487296 00000100:00100000:1.0:1789760793.881737:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772886720933 time 772946495409 state 0x0 ref 1 00000100:00100000:1.0:1789760793.881753:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772946755825 time 772946495409 state 0x1 ref 1 00000100:00100000:1.0:1789760793.881787:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772946755825 time 772986497278 state 0x0 ref 1 00000100:00100000:1.0:1789760793.881801:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508487296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760793.881854:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25587712, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:1.0:1789760793.881864:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe571e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.881880:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=772s v=(0 0) 00010000:00010000:1.0:1789760793.881889:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.881896:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe571e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.881910:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe571e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.916202:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973567 is committed 00002000:00010000:3.0:1789760793.916374:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25587712 -> 25591808 00010000:00010000:3.0:1789760793.916395:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe571e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.916417:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe571e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.916428:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe571e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.916436:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760793.916441:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe571e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.916452:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe571e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.916480:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe571e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.916488:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe571e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25587712->25591807] (req 25587712->25591807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.916624:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508487296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34815us (35201us total) trans 4294973567 rc 0/0 00000100:00100000:3.0:1789760793.916637:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6279 00000100:00100000:3.0:1789760793.916648:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772946755825 time 772986497278 state 0x2 ref 0 00000100:00100000:0.0:1789760793.920233:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760793.920504:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508487424 00000100:00100000:3.0:1789760793.920518:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508487424 00000100:00100000:3.0:1789760793.920661:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772946755825 time 772986497278 state 0x0 ref 1 00000100:00100000:3.0:1789760793.920693:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772986757694 time 772986497278 state 0x1 ref 1 00000100:00100000:3.0:1789760793.921634:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772986757694 time 773026338130 state 0x0 ref 1 00000100:00100000:3.0:1789760793.921658:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508487424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760793.921740:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25591808, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:3.0:1789760793.921751:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.921771:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.921776:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.921783:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.921802:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.955717:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973568 is committed 00002000:00010000:3.0:1789760793.955786:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25591808 -> 25595904 00010000:00010000:3.0:1789760793.955810:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.955827:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.955843:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.955851:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760793.955857:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.955866:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.955893:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.955901:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25591808->25595903] (req 25591808->25595903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.956046:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508487424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34386us (35817us total) trans 4294973568 rc 0/0 00000100:00100000:3.0:1789760793.956058:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6280 00000100:00100000:3.0:1789760793.956070:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772986757694 time 773026338130 state 0x2 ref 0 00000100:00100000:0.0:1789760793.962421:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760793.962528:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508487552 00000100:00100000:3.0:1789760793.962542:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508487552 00000100:00100000:3.0:1789760793.962614:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 772986757694 time 773026338130 state 0x0 ref 1 00000100:00100000:3.0:1789760793.962755:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773026598546 time 773026338130 state 0x1 ref 1 00000100:00100000:3.0:1789760793.962831:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773026598546 time 773067540047 state 0x0 ref 1 00000100:00100000:3.0:1789760793.962846:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508487552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760793.962909:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25595904, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:3.0:1789760793.962920:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.962938:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760793.962944:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.962952:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.962968:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.002108:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973569 is committed 00002000:00010000:3.0:1789760794.002219:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25595904 -> 25600000 00010000:00010000:3.0:1789760794.002238:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.002254:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.002264:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.002272:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760794.002279:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.002315:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.002341:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.002350:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25595904->25599999] (req 25595904->25599999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.002460:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987255f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508487552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39611us (40047us total) trans 4294973569 rc 0/0 00000100:00100000:3.0:1789760794.002470:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6281 00000100:00100000:3.0:1789760794.002482:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773026598546 time 773067540047 state 0x2 ref 0 00000100:00100000:0.0:1789760794.006975:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760794.007141:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508487680 00000100:00100000:3.0:1789760794.007154:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508487680 00000100:00100000:3.0:1789760794.007224:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773026598546 time 773067540047 state 0x0 ref 1 00000100:00100000:3.0:1789760794.007242:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773067800463 time 773067540047 state 0x1 ref 1 00000100:00100000:3.0:1789760794.007364:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773067800463 time 773112073864 state 0x0 ref 1 00000100:00100000:3.0:1789760794.007378:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508487680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760794.007445:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25600000, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:3.0:1789760794.007456:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe571f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.007476:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.007481:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.007488:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe571f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.007504:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe571f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.067843:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973570 is committed 00002000:00010000:3.0:1789760794.067961:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25600000 -> 25604096 00010000:00010000:3.0:1789760794.067980:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe571f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.067995:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe571f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.068005:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe571f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.068012:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760794.068018:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe571f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.068088:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe571f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.068120:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe571f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.068128:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe571f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25600000->25604095] (req 25600000->25604095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.068239:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04987256d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508487680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60857us (61273us total) trans 4294973570 rc 0/0 00000100:00100000:3.0:1789760794.068250:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6282 00000100:00100000:3.0:1789760794.068262:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773067800463 time 773112073864 state 0x2 ref 0 00000100:00100000:0.0:1789760794.071696:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760794.071776:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508487808 00000100:00100000:3.0:1789760794.071789:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508487808 00000100:00100000:3.0:1789760794.071865:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773067800463 time 773112073864 state 0x0 ref 1 00000100:00100000:3.0:1789760794.071889:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773112334280 time 773112073864 state 0x1 ref 1 00000100:00100000:3.0:1789760794.071920:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773112334280 time 773176630576 state 0x0 ref 1 00000100:00100000:3.0:1789760794.071934:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508487808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760794.072003:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25604096, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:3.0:1789760794.072014:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.072102:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.072109:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.072116:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.072133:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760794.107543:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25604096 -> 25608192 00010000:00010000:3.0:1789760794.107573:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.107588:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.107609:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.107618:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760794.107624:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.107634:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.107660:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.107667:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe571fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25604096->25608191] (req 25604096->25608191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.107806:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7342680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508487808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35867us (36128us total) trans 4294973571 rc 0/0 00000100:00100000:3.0:1789760794.107819:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6283 00000100:00100000:3.0:1789760794.107833:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773112334280 time 773176630576 state 0x2 ref 0 00000001:00080000:0.0:1789760794.109338:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973571 is committed 00000100:00100000:0.0:1789760794.111546:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760794.111667:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508487936 00000100:00100000:3.0:1789760794.111688:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508487936 00000100:00100000:3.0:1789760794.111757:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773112334280 time 773176630576 state 0x0 ref 1 00000100:00100000:3.0:1789760794.111774:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773176890992 time 773176630576 state 0x1 ref 1 00000100:00100000:3.0:1789760794.111808:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773176890992 time 773216518248 state 0x0 ref 1 00000100:00100000:3.0:1789760794.111820:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508487936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760794.111879:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25608192, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:3.0:1789760794.111889:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57203 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.111909:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.111914:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.111922:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57203 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.111939:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57203 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.138520:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973572 is committed 00002000:00010000:3.0:1789760794.138521:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25608192 -> 25612288 00010000:00010000:3.0:1789760794.138544:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57203 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.138558:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57203 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.138569:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57203 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.138576:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760794.138582:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57203 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.138592:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57203 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.138617:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57203 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.138626:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57203 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25608192->25612287] (req 25608192->25612287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.139074:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508487936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27245us (27531us total) trans 4294973572 rc 0/0 00000100:00100000:3.0:1789760794.139090:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6284 00000100:00100000:3.0:1789760794.139103:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773176890992 time 773216518248 state 0x2 ref 0 00000100:00100000:2.0:1789760794.142791:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760794.142889:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508488064 00000100:00100000:3.0:1789760794.142903:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508488064 00000100:00100000:3.0:1789760794.142978:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773176890992 time 773216518248 state 0x0 ref 1 00000100:00100000:3.0:1789760794.142997:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773216778664 time 773216518248 state 0x1 ref 1 00000100:00100000:3.0:1789760794.143031:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773216778664 time 773247741386 state 0x0 ref 1 00000100:00100000:3.0:1789760794.143047:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508488064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760794.143118:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25612288, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:3.0:1789760794.143129:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5720a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.143160:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.143165:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.143173:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5720a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.143190:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5720a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760794.174016:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25612288 -> 25616384 00010000:00010000:2.0:1789760794.174048:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5720a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.174064:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5720a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.174073:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5720a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.174082:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760794.174087:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5720a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.174096:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5720a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.174121:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5720a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.174129:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005da8cca/0xb49177c0efe5720a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25612288->25616383] (req 25612288->25616383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.174263:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508488064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31212us (31480us total) trans 4294973573 rc 0/0 00000100:00100000:2.0:1789760794.174275:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6285 00000100:00100000:2.0:1789760794.174330:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773216778664 time 773247741386 state 0x2 ref 0 00000001:00080000:3.0:1789760794.174970:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973573 is committed 00000100:00100000:0.0:1789760794.178586:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760794.180604:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508488192 00000100:00100000:3.0:1789760794.180625:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508488192 00000100:00100000:2.0:1789760794.180879:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773216778664 time 773247741386 state 0x0 ref 1 00000100:00100000:2.0:1789760794.180899:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773248001802 time 773247741386 state 0x1 ref 1 00000100:00100000:2.0:1789760794.180993:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773248001802 time 773285701883 state 0x0 ref 1 00000100:00100000:2.0:1789760794.181010:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508488192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760794.181079:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25616384, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:2.0:1789760794.181090:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57211 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.181110:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760794.181116:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.181124:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57211 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.181143:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57211 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.235325:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973574 is committed 00002000:00010000:2.0:1789760794.235503:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25616384 -> 25620480 00010000:00010000:2.0:1789760794.235525:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57211 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.235541:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57211 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.235551:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57211 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.235559:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760794.235565:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57211 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.235576:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57211 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.235603:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57211 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.235611:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57211 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25616384->25620479] (req 25616384->25620479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.235847:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508488192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54834us (57266us total) trans 4294973574 rc 0/0 00000100:00100000:2.0:1789760794.235862:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6286 00000100:00100000:2.0:1789760794.235874:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773248001802 time 773285701883 state 0x2 ref 0 00000100:00100000:1.0:1789760794.241491:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760794.241599:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508488320 00000100:00100000:3.0:1789760794.241613:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508488320 00000100:00100000:3.0:1789760794.241695:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773248001802 time 773285701883 state 0x0 ref 1 00000100:00100000:3.0:1789760794.241715:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773285962299 time 773285701883 state 0x1 ref 1 00000100:00100000:3.0:1789760794.241755:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773285962299 time 773346464788 state 0x0 ref 1 00000100:00100000:3.0:1789760794.241770:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888133100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508488320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760794.241835:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25620480, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:3.0:1789760794.241845:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57218 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.242069:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.242075:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.242084:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57218 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.242105:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57218 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.282035:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973575 is committed 00002000:00010000:3.0:1789760794.282248:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25620480 -> 25624576 00010000:00010000:3.0:1789760794.282268:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57218 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.282348:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57218 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.282361:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57218 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.282369:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760794.282375:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57218 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.282385:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57218 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.282409:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57218 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.282416:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe57218 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25620480->25624575] (req 25620480->25624575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.282607:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888133100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508488320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40832us (41123us total) trans 4294973575 rc 0/0 00000100:00100000:2.0:1789760794.282619:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6287 00000100:00100000:2.0:1789760794.282631:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773285962299 time 773346464788 state 0x2 ref 0 00000100:00100000:1.0:1789760794.286056:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760794.286241:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508488448 00000100:00100000:2.0:1789760794.286257:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508488448 00000100:00100000:2.0:1789760794.286425:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773285962299 time 773346464788 state 0x0 ref 1 00000100:00100000:2.0:1789760794.286451:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773346725204 time 773346464788 state 0x1 ref 1 00000100:00100000:2.0:1789760794.286524:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773346725204 time 773391233522 state 0x0 ref 1 00000100:00100000:2.0:1789760794.286538:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508488448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760794.286602:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25624576, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:2.0:1789760794.286612:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5721f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.286632:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760794.286637:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.286655:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5721f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.286676:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5721f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.335798:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973576 is committed 00002000:00010000:2.0:1789760794.335996:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25624576 -> 25628672 00010000:00010000:2.0:1789760794.336020:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5721f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.336036:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5721f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.336047:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5721f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.336055:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760794.336062:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5721f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.336073:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5721f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.336100:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5721f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.336107:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe5721f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25624576->25628671] (req 25624576->25628671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.336369:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888133b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508488448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49826us (50321us total) trans 4294973576 rc 0/0 00000100:00100000:2.0:1789760794.336382:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6288 00000100:00100000:2.0:1789760794.336393:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773346725204 time 773391233522 state 0x2 ref 0 00000100:00100000:0.0:1789760794.341242:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760794.341584:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508488576 00000100:00100000:2.0:1789760794.341597:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508488576 00000100:00100000:2.0:1789760794.341673:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773346725204 time 773391233522 state 0x0 ref 1 00000100:00100000:2.0:1789760794.341701:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773391493938 time 773391233522 state 0x1 ref 1 00000100:00100000:2.0:1789760794.341725:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773391493938 time 773446435300 state 0x0 ref 1 00000100:00100000:2.0:1789760794.341737:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508488576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760794.341794:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25628672, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:2.0:1789760794.341805:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57226 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.341825:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760794.341830:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.341839:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57226 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.341858:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57226 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.369316:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760794.403852:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973577 is committed 00002000:00010000:0.0:1789760794.404000:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25628672 -> 25632768 00010000:00010000:0.0:1789760794.404020:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57226 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.404045:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57226 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.404056:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57226 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.404064:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760794.404069:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57226 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.404081:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57226 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.404107:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57226 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.404115:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000085104744/0xb49177c0efe57226 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25628672->25632767] (req 25628672->25632767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.404393:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049b7340a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508488576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 62650us (63157us total) trans 4294973577 rc 0/0 00000100:00100000:0.0:1789760794.404413:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6289 00000100:00100000:0.0:1789760794.404427:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773391493938 time 773446435300 state 0x2 ref 0 00000100:00100000:1.0:1789760794.411004:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760794.416777:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508488704 00000100:00100000:2.0:1789760794.416802:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508488704 00000100:00100000:3.0:1789760794.416972:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773391493938 time 773446435300 state 0x0 ref 1 00000100:00100000:3.0:1789760794.416994:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773446695716 time 773446435300 state 0x1 ref 1 00000100:00100000:3.0:1789760794.417156:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773446695716 time 773521864999 state 0x0 ref 1 00000100:00100000:3.0:1789760794.417174:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508488704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760794.417240:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25632768, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:3.0:1789760794.417251:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5722d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.417274:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:3.0:1789760794.417279:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.417305:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5722d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.417323:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5722d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.473941:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973578 is committed 00002000:00010000:0.0:1789760794.474164:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25632768 -> 25636864 00010000:00010000:0.0:1789760794.474178:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5722d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.474192:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5722d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.474201:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5722d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.474209:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760794.474213:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5722d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.474222:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5722d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.474241:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5722d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.474248:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6ed0adf/0xb49177c0efe5722d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25632768->25636863] (req 25632768->25636863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.474386:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888132680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508488704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57211us (63393us total) trans 4294973578 rc 0/0 00000100:00100000:0.0:1789760794.474400:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6290 00000100:00100000:0.0:1789760794.474409:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773446695716 time 773521864999 state 0x2 ref 0 00000100:00100000:1.0:1789760794.479112:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760794.484352:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508488832 00000100:00100000:2.0:1789760794.484368:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508488832 00000100:00100000:2.0:1789760794.484498:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773446695716 time 773521864999 state 0x0 ref 1 00000100:00100000:2.0:1789760794.484517:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773522125415 time 773521864999 state 0x1 ref 1 00000100:00100000:2.0:1789760794.484596:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773522125415 time 773589305583 state 0x0 ref 1 00000100:00100000:2.0:1789760794.484612:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508488832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760794.484662:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25636864, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0xc807 00010000:00010000:2.0:1789760794.484672:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57234 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.484699:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:2.0:1789760794.484704:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.484710:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57234 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.484723:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57234 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.530122:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973579 is committed 00002000:00010000:0.0:1789760794.530378:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25636864 -> 25640960 00002000:00010000:0.0:1789760794.530385:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760794 -> 1789760795 00002000:00010000:0.0:1789760794.530388:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760794 -> 1789760795 00002000:00010000:0.0:1789760794.530391:0:8108:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 51207 -> 51209 00010000:00010000:0.0:1789760794.530410:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57234 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.530426:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57234 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.530436:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57234 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.530445:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760794.530450:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57234 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.530462:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57234 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.530489:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57234 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.530503:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57234 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25636864->25640959] (req 25636864->25640959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.530623:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888131f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508488832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46008us (51513us total) trans 4294973579 rc 0/0 00000100:00100000:0.0:1789760794.530637:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6291 00000100:00100000:0.0:1789760794.530649:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773522125415 time 773589305583 state 0x2 ref 0 00000100:00100000:2.0:1789760794.535608:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760794.535717:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508488960 00000100:00100000:0.0:1789760794.535732:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508488960 00000100:00100000:0.0:1789760794.535819:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773522125415 time 773589305583 state 0x0 ref 1 00000100:00100000:0.0:1789760794.535840:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773589565999 time 773589305583 state 0x1 ref 1 00000100:00100000:0.0:1789760794.535940:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773589565999 time 773640648777 state 0x0 ref 1 00000100:00100000:0.0:1789760794.535956:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508488960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760794.536026:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25640960, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:0.0:1789760794.536037:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5723b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.536056:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760794.536061:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.536069:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5723b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.536091:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5723b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760794.564422:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25640960 -> 25645056 00000001:00080000:3.0:1789760794.564440:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973580 is committed 00010000:00010000:1.0:1789760794.564538:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5723b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.564552:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5723b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.564561:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5723b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.564568:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760794.564574:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5723b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.564584:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5723b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.564611:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5723b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.564618:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe5723b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25640960->25645055] (req 25640960->25645055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.565095:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508488960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28830us (29192us total) trans 4294973580 rc 0/0 00000100:00100000:1.0:1789760794.565111:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6292 00000100:00100000:1.0:1789760794.565124:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773589565999 time 773640648777 state 0x2 ref 0 00000100:00100000:1.0:1789760794.569501:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760794.569584:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508489088 00000100:00100000:0.0:1789760794.569609:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508489088 00000100:00100000:0.0:1789760794.569695:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773589565999 time 773640648777 state 0x0 ref 1 00000100:00100000:0.0:1789760794.569715:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773640909193 time 773640648777 state 0x1 ref 1 00000100:00100000:0.0:1789760794.569754:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773640909193 time 773674464305 state 0x0 ref 1 00000100:00100000:0.0:1789760794.569767:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508489088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760794.569831:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25645056, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:0.0:1789760794.569841:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57242 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.569861:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760794.569867:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.569874:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.569891:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.595423:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973581 is committed 00002000:00010000:0.0:1789760794.595642:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25645056 -> 25649152 00010000:00010000:0.0:1789760794.595662:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.595685:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57242 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.595695:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57242 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.595703:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760794.595708:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57242 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.595718:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57242 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.595743:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57242 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.595752:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57242 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25645056->25649151] (req 25645056->25649151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.595878:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508489088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26105us (26384us total) trans 4294973581 rc 0/0 00000100:00100000:0.0:1789760794.595890:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6293 00000100:00100000:0.0:1789760794.595902:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773640909193 time 773674464305 state 0x2 ref 0 00000100:00100000:2.0:1789760794.598716:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760794.598767:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508489216 00000100:00100000:0.0:1789760794.598781:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508489216 00000100:00100000:0.0:1789760794.598855:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773640909193 time 773674464305 state 0x0 ref 1 00000100:00100000:0.0:1789760794.598875:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773674724721 time 773674464305 state 0x1 ref 1 00000100:00100000:0.0:1789760794.598911:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773674724721 time 773703622005 state 0x0 ref 1 00000100:00100000:0.0:1789760794.598924:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508489216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760794.598988:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25649152, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:0.0:1789760794.598999:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57249 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.599019:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760794.599037:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.599044:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57249 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.599060:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57249 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.668432:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973582 is committed 00002000:00010000:1.0:1789760794.668590:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25649152 -> 25653248 00010000:00010000:1.0:1789760794.668610:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57249 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.668623:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57249 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.668633:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57249 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.668642:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760794.668647:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57249 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.668658:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57249 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.668697:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57249 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.668707:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57249 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25649152->25653247] (req 25649152->25653247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.668897:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508489216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 69968us (70199us total) trans 4294973582 rc 0/0 00000100:00100000:1.0:1789760794.668911:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6294 00000100:00100000:1.0:1789760794.668923:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773674724721 time 773703622005 state 0x2 ref 0 00000100:00100000:1.0:1789760794.677448:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760794.677649:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508489344 00000100:00100000:0.0:1789760794.677672:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508489344 00000100:00100000:0.0:1789760794.677757:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773674724721 time 773703622005 state 0x0 ref 1 00000100:00100000:0.0:1789760794.677775:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773703882421 time 773703622005 state 0x1 ref 1 00000100:00100000:0.0:1789760794.677817:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773703882421 time 773782526766 state 0x0 ref 1 00000100:00100000:0.0:1789760794.677832:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508489344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760794.677901:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25653248, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:0.0:1789760794.677911:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57250 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.677931:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760794.677936:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.677943:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57250 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.677960:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57250 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760794.727614:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25653248 -> 25657344 00010000:00010000:0.0:1789760794.727645:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57250 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.727660:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57250 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.727671:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57250 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.727689:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760794.727695:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57250 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.727706:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57250 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.727732:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57250 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.727740:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57250 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25653248->25657343] (req 25653248->25657343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.727868:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508489344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50031us (50428us total) trans 4294973583 rc 0/0 00000100:00100000:0.0:1789760794.727881:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6295 00000100:00100000:0.0:1789760794.727903:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773703882421 time 773782526766 state 0x2 ref 0 00000001:00080000:3.0:1789760794.728965:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973583 is committed 00000100:00100000:0.0:1789760794.739998:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760794.742432:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508489472 00000100:00100000:2.0:1789760794.742453:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508489472 00000100:00100000:3.0:1789760794.747235:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773703882421 time 773782526766 state 0x0 ref 1 00000100:00100000:3.0:1789760794.747263:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773782787182 time 773782526766 state 0x1 ref 1 00000100:00100000:0.0:1789760794.747448:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773782787182 time 773852155114 state 0x0 ref 1 00000100:00100000:0.0:1789760794.747470:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508489472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760794.747542:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25657344, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:0.0:1789760794.747554:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57257 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747587:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:0.0:1789760794.747594:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.747602:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57257 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.747619:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57257 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.812328:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973584 is committed 00002000:00010000:0.0:1789760794.812480:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25657344 -> 25661440 00010000:00010000:0.0:1789760794.812499:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57257 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.812514:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57257 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.812525:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57257 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.812533:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760794.812539:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57257 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.812550:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57257 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.812576:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57257 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.812589:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57257 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25657344->25661439] (req 25657344->25661439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.813341:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508489472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 65822us (73299us total) trans 4294973584 rc 0/0 00000100:00100000:0.0:1789760794.813354:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6296 00000100:00100000:0.0:1789760794.813366:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773782787182 time 773852155114 state 0x2 ref 0 00000100:00100000:2.0:1789760794.823522:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760794.823639:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508489600 00000100:00100000:1.0:1789760794.823653:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508489600 00000100:00100000:1.0:1789760794.823743:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773782787182 time 773852155114 state 0x0 ref 1 00000100:00100000:1.0:1789760794.823763:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773852415530 time 773852155114 state 0x1 ref 1 00000100:00100000:1.0:1789760794.823802:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773852415530 time 773928512444 state 0x0 ref 1 00000100:00100000:1.0:1789760794.823817:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508489600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760794.823893:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25661440, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:1.0:1789760794.823904:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5725e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.823923:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=773s v=(0 0) 00010000:00010000:1.0:1789760794.823928:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.823936:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5725e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.823953:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5725e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760794.905277:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25661440 -> 25665536 00010000:00010000:3.0:1789760794.905334:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5725e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.905349:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5725e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.905363:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5725e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.905370:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760794.905376:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5725e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.905385:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5725e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.905418:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5725e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.905427:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe5725e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25661440->25665535] (req 25661440->25665535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.905553:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508489600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 81730us (82039us total) trans 4294973585 rc 0/0 00000100:00100000:3.0:1789760794.905564:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6297 00000100:00100000:3.0:1789760794.905577:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773852415530 time 773928512444 state 0x2 ref 0 00000001:00080000:1.0:1789760794.906592:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973585 is committed 00000100:00100000:1.0:1789760794.915010:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760794.915176:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508489728 00000100:00100000:0.0:1789760794.915189:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508489728 00000100:00100000:0.0:1789760794.915283:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773852415530 time 773928512444 state 0x0 ref 1 00000100:00100000:0.0:1789760794.915349:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773928772860 time 773928512444 state 0x1 ref 1 00000100:00100000:0.0:1789760794.915375:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773928772860 time 774020085404 state 0x0 ref 1 00000100:00100000:0.0:1789760794.915390:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c1fcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508489728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760794.915460:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25665536, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:0.0:1789760794.915472:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57265 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.915492:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760794.915497:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.915505:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.915527:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.966795:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973586 is committed 00002000:00010000:2.0:1789760794.967003:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25665536 -> 25669632 00010000:00010000:2.0:1789760794.967020:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.967099:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57265 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.967107:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57265 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.967112:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760794.967117:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57265 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.967124:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57265 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.967145:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57265 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.967150:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57265 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25665536->25669631] (req 25665536->25669631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.967261:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c1fcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508489728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51867us (52260us total) trans 4294973586 rc 0/0 00000100:00100000:2.0:1789760794.967273:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6298 00000100:00100000:2.0:1789760794.967329:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773928772860 time 774020085404 state 0x2 ref 0 00000100:00100000:0.0:1789760794.972709:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760794.973215:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508489856 00000100:00100000:2.0:1789760794.973228:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508489856 00000100:00100000:2.0:1789760794.973411:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 773928772860 time 774020085404 state 0x0 ref 1 00000100:00100000:2.0:1789760794.973432:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774020345820 time 774020085404 state 0x1 ref 1 00000100:00100000:2.0:1789760794.973459:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774020345820 time 774078169406 state 0x0 ref 1 00000100:00100000:2.0:1789760794.973474:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e61180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508489856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760794.973540:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25669632, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:2.0:1789760794.973552:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5726c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.973841:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760794.973849:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.973857:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5726c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.973876:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5726c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.037827:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973587 is committed 00002000:00010000:0.0:1789760795.042140:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25669632 -> 25673728 00010000:00010000:0.0:1789760795.042325:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5726c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.042340:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5726c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.042350:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5726c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.042360:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760795.042365:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5726c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.042375:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5726c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.042399:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5726c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.042407:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe5726c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25669632->25673727] (req 25669632->25673727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.042644:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e61180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508489856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 69164us (69940us total) trans 4294973587 rc 0/0 00000100:00100000:1.0:1789760795.042658:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6299 00000100:00100000:1.0:1789760795.042671:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774020345820 time 774078169406 state 0x2 ref 0 00000100:00100000:1.0:1789760795.046653:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760795.048370:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508489984 00000100:00100000:3.0:1789760795.048390:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508489984 00000100:00100000:3.0:1789760795.048505:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774020345820 time 774078169406 state 0x0 ref 1 00000100:00080000:2.0:1789760795.048507:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760795.048531:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774078429822 time 774078169406 state 0x1 ref 1 00000100:00100000:3.0:1789760795.048572:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774078429822 time 774153281755 state 0x0 ref 1 00000100:00100000:3.0:1789760795.048596:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508489984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760795.049097:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25673728, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:3.0:1789760795.049111:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57273 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.049132:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760795.049137:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.049146:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57273 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.049163:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57273 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.122625:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973588 is committed 00002000:00010000:1.0:1789760795.123055:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25673728 -> 25677824 00010000:00010000:1.0:1789760795.123084:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57273 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.123393:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57273 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.123409:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57273 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.123418:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760795.123424:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57273 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.123436:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57273 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.123470:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57273 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.123480:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57273 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25673728->25677823] (req 25673728->25677823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.123742:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508489984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 75142us (77096us total) trans 4294973588 rc 0/0 00000100:00100000:1.0:1789760795.123756:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6300 00000100:00100000:1.0:1789760795.123770:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774078429822 time 774153281755 state 0x2 ref 0 00000100:00100000:2.0:1789760795.128450:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760795.128756:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508490112 00000100:00100000:1.0:1789760795.128770:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508490112 00000100:00100000:1.0:1789760795.128842:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774078429822 time 774153281755 state 0x0 ref 1 00000100:00100000:1.0:1789760795.128862:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774153542171 time 774153281755 state 0x1 ref 1 00000100:00100000:1.0:1789760795.128887:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774153542171 time 774233597111 state 0x0 ref 1 00000100:00100000:1.0:1789760795.128900:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508490112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760795.128963:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25677824, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:1.0:1789760795.128974:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5727a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.129633:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760795.134892:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.134903:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5727a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.134923:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5727a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.177845:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973589 is committed 00002000:00010000:3.0:1789760795.177976:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25677824 -> 25681920 00010000:00010000:3.0:1789760795.177998:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5727a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.178015:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5727a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.178160:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5727a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.178178:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760795.178184:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5727a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.178195:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5727a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.178223:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5727a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.178231:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5727a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25677824->25681919] (req 25677824->25681919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.178444:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508490112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49539us (50004us total) trans 4294973589 rc 0/0 00000100:00100000:3.0:1789760795.178456:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6301 00000100:00100000:3.0:1789760795.178478:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774153542171 time 774233597111 state 0x2 ref 0 00000100:00100000:0.0:1789760795.182598:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760795.182746:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508490240 00000100:00100000:1.0:1789760795.182759:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508490240 00000100:00100000:1.0:1789760795.182831:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774153542171 time 774233597111 state 0x0 ref 1 00000100:00100000:1.0:1789760795.182850:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774233857527 time 774233597111 state 0x1 ref 1 00000100:00100000:1.0:1789760795.182889:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774233857527 time 774287599006 state 0x0 ref 1 00000100:00100000:1.0:1789760795.182902:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e63b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508490240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760795.182967:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25681920, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:1.0:1789760795.182977:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57281 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.182996:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760795.183000:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.183008:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57281 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.183035:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57281 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760795.238381:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25681920 -> 25686016 00010000:00010000:3.0:1789760795.238409:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57281 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.238425:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57281 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.238442:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57281 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.238450:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760795.238456:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57281 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.238467:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57281 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.238493:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57281 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.238502:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57281 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25681920->25686015] (req 25681920->25686015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.238639:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e63b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508490240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55731us (56060us total) trans 4294973590 rc 0/0 00000100:00100000:3.0:1789760795.238661:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6302 00000100:00100000:3.0:1789760795.238683:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774233857527 time 774287599006 state 0x2 ref 0 00000001:00080000:0.0:1789760795.239793:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973590 is committed 00000100:00100000:1.0:1789760795.247481:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760795.247579:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508490368 00000100:00100000:2.0:1789760795.247592:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508490368 00000100:00100000:2.0:1789760795.247666:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774233857527 time 774287599006 state 0x0 ref 1 00000100:00100000:2.0:1789760795.247696:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774287859422 time 774287599006 state 0x1 ref 1 00000100:00100000:2.0:1789760795.247769:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774287859422 time 774352479348 state 0x0 ref 1 00000100:00100000:2.0:1789760795.247784:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508490368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760795.247847:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25686016, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:2.0:1789760795.247857:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57288 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.247877:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760795.247882:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.247890:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57288 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.247906:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57288 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.294212:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973591 is committed 00002000:00010000:2.0:1789760795.294558:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25686016 -> 25690112 00010000:00010000:2.0:1789760795.294580:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57288 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.294595:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57288 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.294604:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57288 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.294613:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760795.294618:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57288 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.294628:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57288 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.294653:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57288 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.294662:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57288 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25686016->25690111] (req 25686016->25690111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.295965:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508490368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47371us (47983us total) trans 4294973591 rc 0/0 00000100:00100000:3.0:1789760795.295981:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6303 00000100:00100000:3.0:1789760795.295993:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774287859422 time 774352479348 state 0x2 ref 0 00000100:00100000:1.0:1789760795.300232:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760795.300437:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508490496 00000100:00100000:3.0:1789760795.300452:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508490496 00000100:00100000:3.0:1789760795.300678:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774287859422 time 774352479348 state 0x0 ref 1 00000100:00100000:3.0:1789760795.300700:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774352739764 time 774352479348 state 0x1 ref 1 00000100:00100000:3.0:1789760795.300838:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774352739764 time 774405546463 state 0x0 ref 1 00000100:00100000:3.0:1789760795.300853:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508490496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760795.300917:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25690112, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc807 00010000:00010000:3.0:1789760795.300927:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5728f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.300946:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760795.300952:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.300960:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5728f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.300976:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5728f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760795.350502:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25690112 -> 25694208 00002000:00010000:0.0:1789760795.350514:0:8109:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 51207 -> 51209 00010000:00010000:0.0:1789760795.350531:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5728f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.350548:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5728f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.350558:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5728f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.350566:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760795.350572:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5728f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.350590:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5728f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.350617:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5728f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.350625:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5728f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25690112->25694207] (req 25690112->25694207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.350928:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508490496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50070us (50704us total) trans 4294973592 rc 0/0 00000100:00100000:0.0:1789760795.350938:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6304 00000100:00100000:0.0:1789760795.350949:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774352739764 time 774405546463 state 0x2 ref 0 00000100:00100000:0.0:1789760795.354573:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760795.354841:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508490624 00000100:00100000:3.0:1789760795.354854:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508490624 00000100:00100000:3.0:1789760795.354932:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774352739764 time 774405546463 state 0x0 ref 1 00000100:00100000:3.0:1789760795.354952:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774405806879 time 774405546463 state 0x1 ref 1 00000100:00100000:3.0:1789760795.354990:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774405806879 time 774459700080 state 0x0 ref 1 00000100:00100000:3.0:1789760795.355005:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508490624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760795.355089:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25694208, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:3.0:1789760795.355101:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57296 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.355122:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760795.355127:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.355135:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57296 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.355152:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57296 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.356732:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973592 is committed 00000001:00080000:2.0:1789760795.391189:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973593 is committed 00002000:00010000:3.0:1789760795.391406:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25694208 -> 25698304 00010000:00010000:3.0:1789760795.391427:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57296 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.391441:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57296 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.391451:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57296 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.391460:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760795.391466:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57296 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.391477:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57296 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.391503:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57296 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.391511:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57296 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25694208->25698303] (req 25694208->25698303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.391691:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508490624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 36680us (37125us total) trans 4294973593 rc 0/0 00000100:00100000:3.0:1789760795.391772:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6305 00000100:00100000:3.0:1789760795.391786:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774405806879 time 774459700080 state 0x2 ref 0 00010000:00010000:3.0:1789760795.391875:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760795.398112:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760795.398229:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508490752 00000100:00100000:1.0:1789760795.398248:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508490752 00000100:00100000:1.0:1789760795.398361:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774405806879 time 774459700080 state 0x0 ref 1 00000100:00100000:1.0:1789760795.398381:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774459960496 time 774459700080 state 0x1 ref 1 00000100:00100000:1.0:1789760795.398416:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774459960496 time 774503126991 state 0x0 ref 1 00000100:00100000:1.0:1789760795.398431:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508490752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760795.398507:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25698304, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:1.0:1789760795.398518:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5729d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.398540:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760795.398546:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.398554:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5729d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.398571:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5729d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.448867:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973594 is committed 00002000:00010000:2.0:1789760795.449113:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25698304 -> 25702400 00010000:00010000:2.0:1789760795.449134:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5729d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.449155:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5729d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.449165:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5729d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.449172:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760795.449178:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5729d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.449188:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5729d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.449214:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5729d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.449222:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5729d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25698304->25702399] (req 25698304->25702399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.449432:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508490752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50997us (51329us total) trans 4294973594 rc 0/0 00000100:00100000:2.0:1789760795.449446:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6306 00000100:00100000:2.0:1789760795.449459:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774459960496 time 774503126991 state 0x2 ref 0 00000100:00100000:1.0:1789760795.452484:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760795.452593:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508490880 00000100:00100000:0.0:1789760795.452607:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508490880 00000100:00100000:0.0:1789760795.452697:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774459960496 time 774503126991 state 0x0 ref 1 00000100:00100000:0.0:1789760795.452723:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774503387407 time 774503126991 state 0x1 ref 1 00000100:00100000:0.0:1789760795.452758:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774503387407 time 774557468034 state 0x0 ref 1 00000100:00100000:0.0:1789760795.452772:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508490880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760795.452834:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25702400, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:0.0:1789760795.452845:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe572a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.452864:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760795.452869:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.452877:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe572a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.452899:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe572a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.501994:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973595 is committed 00002000:00010000:1.0:1789760795.502122:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25702400 -> 25706496 00010000:00010000:1.0:1789760795.502142:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe572a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.502162:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe572a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.502171:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe572a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.502179:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760795.502184:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe572a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.502195:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe572a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.502222:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe572a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.502232:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe572a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25702400->25706495] (req 25702400->25706495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.502381:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508490880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49605us (49905us total) trans 4294973595 rc 0/0 00000100:00100000:1.0:1789760795.502392:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6307 00000100:00100000:1.0:1789760795.502405:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774503387407 time 774557468034 state 0x2 ref 0 00000100:00100000:1.0:1789760795.507147:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760795.507226:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508491008 00000100:00100000:3.0:1789760795.507240:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508491008 00000100:00100000:3.0:1789760795.507350:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774503387407 time 774557468034 state 0x0 ref 1 00000100:00100000:3.0:1789760795.507371:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774557728450 time 774557468034 state 0x1 ref 1 00000100:00100000:3.0:1789760795.507457:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774557728450 time 774612166723 state 0x0 ref 1 00000100:00100000:3.0:1789760795.507473:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508491008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760795.507540:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25706496, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0xc809 00010000:00010000:3.0:1789760795.507551:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe572ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.507572:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760795.507578:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.507585:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe572ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.507603:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe572ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.518548:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760795.518583:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887fc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534093696:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760795.518946:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760795.519133:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534093696 00000100:00100000:1.0:1789760795.519144:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534093696 00000100:00100000:1.0:1789760795.519424:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:1.0:1789760795.519437:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e63480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534093696:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760795.519494:0:7358:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760795.519564:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e63480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534093696:12345-0@lo:41:osp-pre-1-0.0 Request processed in 130us (617us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760795.519573:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00100000:1.0:1789760795.519641:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887fc8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534093696:0@lo:41:osp-pre-1-0.0 00002000:00010000:0.0:1789760795.552641:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25706496 -> 25710592 00002000:00010000:0.0:1789760795.552663:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760795 -> 1789760796 00002000:00010000:0.0:1789760795.552667:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760795 -> 1789760796 00010000:00010000:0.0:1789760795.552703:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe572ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.552719:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe572ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.552728:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe572ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.552736:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760795.552741:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe572ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.552744:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973596 is committed 00010000:00010000:0.0:1789760795.552751:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe572ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.552780:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe572ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.552788:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe572ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25706496->25710591] (req 25706496->25710591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.552928:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508491008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45449us (45794us total) trans 4294973596 rc 0/0 00000100:00100000:0.0:1789760795.552940:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6308 00000100:00100000:0.0:1789760795.552953:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774557728450 time 774612166723 state 0x2 ref 0 00000100:00100000:3.0:1789760795.556110:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760795.556913:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508491136 00000100:00100000:2.0:1789760795.556930:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508491136 00000100:00100000:2.0:1789760795.557012:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774557728450 time 774612166723 state 0x0 ref 1 00000100:00100000:2.0:1789760795.557034:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774612427139 time 774612166723 state 0x1 ref 1 00000100:00100000:2.0:1789760795.557133:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774612427139 time 774661842294 state 0x0 ref 1 00000100:00100000:2.0:1789760795.557150:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508491136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760795.557221:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25710592, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc809 00010000:00010000:2.0:1789760795.557233:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe572b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.557255:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760795.557259:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.557266:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe572b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.557329:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe572b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760795.617238:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25710592 -> 25714688 00010000:00010000:3.0:1789760795.617266:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe572b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.617283:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe572b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.617328:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe572b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.617335:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760795.617341:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe572b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.617356:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe572b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.617382:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe572b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.617390:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe572b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25710592->25714687] (req 25710592->25714687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.617525:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508491136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60372us (61425us total) trans 4294973597 rc 0/0 00000100:00100000:3.0:1789760795.617537:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6309 00000100:00100000:3.0:1789760795.617549:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774612427139 time 774661842294 state 0x2 ref 0 00000001:00080000:0.0:1789760795.618381:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973597 is committed 00000100:00100000:1.0:1789760795.621100:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760795.621246:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508491264 00000100:00100000:3.0:1789760795.621259:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508491264 00000100:00100000:3.0:1789760795.621374:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774612427139 time 774661842294 state 0x0 ref 1 00000100:00100000:3.0:1789760795.621395:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774662102710 time 774661842294 state 0x1 ref 1 00000100:00100000:3.0:1789760795.621440:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774662102710 time 774726150231 state 0x0 ref 1 00000100:00100000:3.0:1789760795.621454:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508491264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760795.621520:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25714688, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc809 00010000:00010000:3.0:1789760795.621532:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621553:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760795.621558:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.621565:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.621581:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760795.669972:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25714688 -> 25718784 00010000:00010000:3.0:1789760795.670003:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.670030:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.670042:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.670051:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760795.670056:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.670067:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.670097:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.670108:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25714688->25718783] (req 25714688->25718783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.670249:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508491264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48788us (49155us total) trans 4294973598 rc 0/0 00000100:00100000:3.0:1789760795.670261:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6310 00000100:00100000:3.0:1789760795.670274:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774662102710 time 774726150231 state 0x2 ref 0 00000001:00080000:2.0:1789760795.671009:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973598 is committed 00000100:00100000:2.0:1789760795.681248:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760795.682069:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508491392 00000100:00100000:0.0:1789760795.682087:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508491392 00000100:00100000:0.0:1789760795.682179:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774662102710 time 774726150231 state 0x0 ref 1 00000100:00100000:0.0:1789760795.682199:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774726410647 time 774726150231 state 0x1 ref 1 00000100:00100000:0.0:1789760795.682248:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774726410647 time 774786957096 state 0x0 ref 1 00000100:00100000:0.0:1789760795.682261:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508491392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760795.682373:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25718784, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc809 00010000:00010000:0.0:1789760795.682387:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.682415:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:0.0:1789760795.682420:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.682429:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.682448:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.730570:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973599 is committed 00002000:00010000:2.0:1789760795.730779:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25718784 -> 25722880 00010000:00010000:2.0:1789760795.730800:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.730822:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.730831:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.730838:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760795.730844:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.730853:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.730880:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.730889:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25718784->25722879] (req 25718784->25722879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.731089:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508491392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48751us (49781us total) trans 4294973599 rc 0/0 00000100:00100000:2.0:1789760795.731102:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6311 00000100:00100000:2.0:1789760795.731115:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774726410647 time 774786957096 state 0x2 ref 0 00000100:00100000:1.0:1789760795.734739:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760795.738420:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508491520 00000100:00100000:3.0:1789760795.738443:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508491520 00000100:00100000:3.0:1789760795.738537:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774726410647 time 774786957096 state 0x0 ref 1 00000100:00100000:3.0:1789760795.738558:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774787217512 time 774786957096 state 0x1 ref 1 00000100:00100000:3.0:1789760795.738706:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774787217512 time 774843415287 state 0x0 ref 1 00000100:00100000:3.0:1789760795.738724:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508491520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760795.738791:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25722880, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:3.0:1789760795.738802:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe572c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.738824:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:3.0:1789760795.738829:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.738837:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe572c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.738861:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe572c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.779174:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973600 is committed 00002000:00010000:0.0:1789760795.779361:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25722880 -> 25726976 00010000:00010000:0.0:1789760795.779382:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe572c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.779398:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe572c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.779408:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe572c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.779417:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760795.779422:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe572c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.779431:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe572c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.779455:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe572c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.779465:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe572c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25722880->25726975] (req 25722880->25726975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.779824:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508491520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41096us (45091us total) trans 4294973600 rc 0/0 00000100:00100000:0.0:1789760795.779837:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6312 00000100:00100000:0.0:1789760795.779849:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774787217512 time 774843415287 state 0x2 ref 0 00000100:00100000:3.0:1789760795.783029:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760795.784958:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508491648 00000100:00100000:1.0:1789760795.784980:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508491648 00000100:00100000:1.0:1789760795.785064:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774787217512 time 774843415287 state 0x0 ref 1 00000100:00100000:1.0:1789760795.785083:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774843675703 time 774843415287 state 0x1 ref 1 00000100:00100000:1.0:1789760795.785112:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774843675703 time 774889821804 state 0x0 ref 1 00000100:00100000:1.0:1789760795.785128:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508491648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760795.785196:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25726976, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:1.0:1789760795.785207:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe572ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.785239:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:1.0:1789760795.785243:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.785250:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe572ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.785268:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe572ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760795.837884:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25726976 -> 25731072 00010000:00010000:0.0:1789760795.837912:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe572ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.837929:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe572ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.837938:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe572ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.837945:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760795.837951:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe572ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.837961:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe572ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.837997:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe572ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.838006:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe572ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25726976->25731071] (req 25726976->25731071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.838137:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508491648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53006us (55116us total) trans 4294973601 rc 0/0 00000100:00100000:0.0:1789760795.838150:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6313 00000100:00100000:0.0:1789760795.838162:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774843675703 time 774889821804 state 0x2 ref 0 00000001:00080000:2.0:1789760795.838385:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973601 is committed 00000100:00100000:3.0:1789760795.842902:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760795.843209:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508491776 00000100:00100000:2.0:1789760795.843223:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508491776 00000100:00100000:2.0:1789760795.843486:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774843675703 time 774889821804 state 0x0 ref 1 00000100:00100000:2.0:1789760795.843506:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774890082220 time 774889821804 state 0x1 ref 1 00000100:00100000:2.0:1789760795.843603:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774890082220 time 774948312110 state 0x0 ref 1 00000100:00100000:2.0:1789760795.843619:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508491776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760795.843696:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25731072, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:2.0:1789760795.843708:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe572d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.843729:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=774s v=(0 0) 00010000:00010000:2.0:1789760795.843734:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.843742:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe572d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.843759:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe572d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.904722:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973602 is committed 00002000:00010000:2.0:1789760795.904940:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25731072 -> 25735168 00010000:00010000:2.0:1789760795.904962:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe572d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.904977:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe572d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.904989:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe572d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.904998:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760795.905004:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe572d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.905087:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe572d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.905119:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe572d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.905129:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe572d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25731072->25735167] (req 25731072->25735167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.905342:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508491776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 61719us (62448us total) trans 4294973602 rc 0/0 00000100:00100000:2.0:1789760795.905355:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6314 00000100:00100000:2.0:1789760795.905368:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774890082220 time 774948312110 state 0x2 ref 0 00000100:00100000:1.0:1789760795.907922:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760795.908068:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508491904 00000100:00100000:0.0:1789760795.908081:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508491904 00000100:00100000:0.0:1789760795.908222:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774890082220 time 774948312110 state 0x0 ref 1 00000100:00100000:0.0:1789760795.908243:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774948572526 time 774948312110 state 0x1 ref 1 00000100:00100000:0.0:1789760795.908279:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774948572526 time 775012988450 state 0x0 ref 1 00000100:00100000:0.0:1789760795.908329:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508491904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760795.908395:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25735168, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:0.0:1789760795.908405:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe572dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.908428:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760795.908432:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.908439:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe572dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.908455:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe572dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.964558:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973603 is committed 00002000:00010000:1.0:1789760795.964759:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25735168 -> 25739264 00010000:00010000:1.0:1789760795.964780:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe572dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.964794:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe572dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.964803:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe572dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.964810:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760795.964816:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe572dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.964827:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe572dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.964858:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe572dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.964866:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe572dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25735168->25739263] (req 25735168->25739263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.964995:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508491904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 56662us (57080us total) trans 4294973603 rc 0/0 00000100:00100000:1.0:1789760795.965007:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6315 00000100:00100000:1.0:1789760795.965089:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774948572526 time 775012988450 state 0x2 ref 0 00000100:00100000:3.0:1789760795.968757:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760795.968876:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508492032 00000100:00100000:0.0:1789760795.968889:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508492032 00000100:00100000:0.0:1789760795.968964:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 774948572526 time 775012988450 state 0x0 ref 1 00000100:00100000:0.0:1789760795.968983:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775013248866 time 775012988450 state 0x1 ref 1 00000100:00100000:0.0:1789760795.969016:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775013248866 time 775073726103 state 0x0 ref 1 00000100:00100000:0.0:1789760795.969125:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508492032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760795.969385:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25739264, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:0.0:1789760795.969397:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.969417:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760795.969422:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.969431:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.969453:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760796.020427:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25739264 -> 25743360 00010000:00010000:1.0:1789760796.020454:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.020468:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.020478:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.020485:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760796.020491:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.020505:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.020531:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.020539:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe572e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25739264->25743359] (req 25739264->25743359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.020688:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508492032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 51560us (51936us total) trans 4294973604 rc 0/0 00000100:00100000:1.0:1789760796.020701:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6316 00000100:00100000:1.0:1789760796.020713:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775013248866 time 775073726103 state 0x2 ref 0 00000001:00080000:2.0:1789760796.021195:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973604 is committed 00000100:00100000:2.0:1789760796.028463:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760796.028584:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508492160 00000100:00100000:3.0:1789760796.028597:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508492160 00000100:00100000:3.0:1789760796.028682:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775013248866 time 775073726103 state 0x0 ref 1 00000100:00100000:3.0:1789760796.028701:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775073986519 time 775073726103 state 0x1 ref 1 00000100:00100000:3.0:1789760796.028887:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775073986519 time 775133594858 state 0x0 ref 1 00000100:00100000:3.0:1789760796.028903:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508492160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760796.028970:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25743360, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:3.0:1789760796.028981:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.029000:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760796.029005:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.029022:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.029039:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.060738:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973605 is committed 00002000:00010000:3.0:1789760796.060858:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25743360 -> 25747456 00010000:00010000:3.0:1789760796.060876:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.060892:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.060903:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.060911:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760796.060916:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.060926:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.060951:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.061126:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25743360->25747455] (req 25743360->25747455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.061342:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508492160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32435us (32883us total) trans 4294973605 rc 0/0 00000100:00100000:3.0:1789760796.061358:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6317 00000100:00100000:3.0:1789760796.061370:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775073986519 time 775133594858 state 0x2 ref 0 00000100:00100000:0.0:1789760796.065676:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760796.065780:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508492288 00000100:00100000:1.0:1789760796.065794:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508492288 00000100:00100000:1.0:1789760796.065868:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775073986519 time 775133594858 state 0x0 ref 1 00000100:00100000:1.0:1789760796.065889:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775133855274 time 775133594858 state 0x1 ref 1 00000100:00100000:1.0:1789760796.065928:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775133855274 time 775170637759 state 0x0 ref 1 00000100:00100000:1.0:1789760796.065942:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508492288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760796.066014:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25747456, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:1.0:1789760796.066025:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe572f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.067102:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.067109:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.067118:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe572f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.067136:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe572f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760796.115845:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25747456 -> 25751552 00010000:00010000:1.0:1789760796.115874:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe572f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.115888:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe572f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.115897:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe572f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.115905:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760796.115911:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe572f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.115922:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe572f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.115946:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe572f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.115954:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe572f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25747456->25751551] (req 25747456->25751551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.116068:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508492288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50121us (50409us total) trans 4294973606 rc 0/0 00000100:00100000:1.0:1789760796.116079:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6318 00000100:00100000:1.0:1789760796.116091:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775133855274 time 775170637759 state 0x2 ref 0 00000001:00080000:0.0:1789760796.116173:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973606 is committed 00000100:00100000:0.0:1789760796.122254:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760796.124892:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508492416 00000100:00100000:1.0:1789760796.124913:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508492416 00000100:00100000:1.0:1789760796.125003:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775133855274 time 775170637759 state 0x0 ref 1 00000100:00100000:1.0:1789760796.125021:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775170898175 time 775170637759 state 0x1 ref 1 00000100:00100000:1.0:1789760796.125051:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775170898175 time 775229761686 state 0x0 ref 1 00000100:00100000:1.0:1789760796.125065:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508492416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760796.125126:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25751552, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:1.0:1789760796.125137:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe572f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.125157:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.125162:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.125169:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe572f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.125192:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe572f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760796.170810:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25751552 -> 25755648 00010000:00010000:0.0:1789760796.170838:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe572f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.170854:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe572f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.170864:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe572f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.170871:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760796.170876:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe572f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.170885:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe572f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.170909:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe572f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.170918:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe572f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25751552->25755647] (req 25751552->25755647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.171049:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508492416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 45978us (48804us total) trans 4294973607 rc 0/0 00000100:00100000:0.0:1789760796.171060:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6319 00000100:00100000:0.0:1789760796.171072:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775170898175 time 775229761686 state 0x2 ref 0 00000001:00080000:3.0:1789760796.171718:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973607 is committed 00000100:00100000:2.0:1789760796.173777:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760796.174063:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508492544 00000100:00100000:3.0:1789760796.174076:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508492544 00000100:00100000:3.0:1789760796.174151:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775170898175 time 775229761686 state 0x0 ref 1 00000100:00100000:3.0:1789760796.174169:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775230022102 time 775229761686 state 0x1 ref 1 00000100:00100000:3.0:1789760796.174205:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775230022102 time 775278915478 state 0x0 ref 1 00000100:00100000:3.0:1789760796.174220:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508492544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760796.174338:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25755648, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:3.0:1789760796.174349:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.174368:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760796.174375:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.174518:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.174538:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.252222:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973608 is committed 00002000:00010000:0.0:1789760796.252366:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25755648 -> 25759744 00010000:00010000:0.0:1789760796.252387:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.252404:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.252414:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.252422:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760796.252428:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.252439:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.252469:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.252485:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe572ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25755648->25759743] (req 25755648->25759743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.252614:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508492544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 78389us (78844us total) trans 4294973608 rc 0/0 00000100:00100000:0.0:1789760796.252626:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6320 00000100:00100000:0.0:1789760796.252638:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775230022102 time 775278915478 state 0x2 ref 0 00000100:00100000:2.0:1789760796.263002:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760796.263132:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508492672 00000100:00100000:1.0:1789760796.263146:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508492672 00000100:00100000:1.0:1789760796.263221:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775230022102 time 775278915478 state 0x0 ref 1 00000100:00100000:1.0:1789760796.263240:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775279175894 time 775278915478 state 0x1 ref 1 00000100:00100000:1.0:1789760796.263274:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775279175894 time 775367984350 state 0x0 ref 1 00000100:00100000:1.0:1789760796.263325:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508492672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760796.263393:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25759744, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:1.0:1789760796.263405:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57306 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.263426:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.263431:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.263439:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57306 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.263458:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57306 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760796.341730:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25759744 -> 25763840 00010000:00010000:0.0:1789760796.341759:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57306 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.341776:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57306 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.341788:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57306 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.341796:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760796.341802:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57306 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.341819:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57306 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.341845:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57306 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.341853:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57306 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25759744->25763839] (req 25759744->25763839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.341983:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508492672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 78690us (78989us total) trans 4294973609 rc 0/0 00000100:00100000:0.0:1789760796.341996:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6321 00000100:00100000:0.0:1789760796.342008:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775279175894 time 775367984350 state 0x2 ref 0 00000001:00080000:3.0:1789760796.342339:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973609 is committed 00000100:00100000:3.0:1789760796.345494:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760796.345625:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508492800 00000100:00100000:1.0:1789760796.345639:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508492800 00000100:00100000:1.0:1789760796.345725:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775279175894 time 775367984350 state 0x0 ref 1 00000100:00100000:1.0:1789760796.345745:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775368244766 time 775367984350 state 0x1 ref 1 00000100:00100000:1.0:1789760796.345781:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775368244766 time 775450491633 state 0x0 ref 1 00000100:00100000:1.0:1789760796.345794:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508492800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760796.345865:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25763840, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:1.0:1789760796.345876:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5730d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.345897:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.345902:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.345911:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5730d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.345928:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5730d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760796.399455:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25763840 -> 25767936 00010000:00010000:2.0:1789760796.399484:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5730d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.399500:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5730d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.399510:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5730d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.399519:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760796.399525:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5730d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.399535:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5730d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.399563:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5730d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.399571:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5730d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25763840->25767935] (req 25763840->25767935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.399788:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508492800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53988us (54304us total) trans 4294973610 rc 0/0 00000100:00100000:2.0:1789760796.399803:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6322 00000100:00100000:2.0:1789760796.399817:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775368244766 time 775450491633 state 0x2 ref 0 00000001:00080000:3.0:1789760796.399919:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973610 is committed 00000100:00100000:2.0:1789760796.403411:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760796.403510:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508492928 00000100:00100000:0.0:1789760796.403524:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508492928 00000100:00100000:0.0:1789760796.403678:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775368244766 time 775450491633 state 0x0 ref 1 00000100:00100000:0.0:1789760796.403698:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775450752049 time 775450491633 state 0x1 ref 1 00000100:00100000:0.0:1789760796.403739:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775450752049 time 775508448916 state 0x0 ref 1 00000100:00100000:0.0:1789760796.403753:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508492928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760796.403818:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25767936, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:0.0:1789760796.403829:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57314 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.403848:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760796.403853:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.403860:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57314 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.403878:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57314 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.416081:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760796.437322:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973611 is committed 00002000:00010000:0.0:1789760796.437504:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25767936 -> 25772032 00010000:00010000:0.0:1789760796.437534:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57314 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.437549:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57314 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.437560:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57314 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.437569:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760796.437763:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57314 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.437778:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57314 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.437804:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57314 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.437813:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57314 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25767936->25772031] (req 25767936->25772031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.437944:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508492928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34185us (34540us total) trans 4294973611 rc 0/0 00000100:00100000:0.0:1789760796.437956:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6323 00000100:00100000:0.0:1789760796.437968:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775450752049 time 775508448916 state 0x2 ref 0 00000100:00100000:3.0:1789760796.442140:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760796.442534:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508493056 00000100:00100000:2.0:1789760796.442547:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508493056 00000100:00100000:2.0:1789760796.442695:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775450752049 time 775508448916 state 0x0 ref 1 00000100:00100000:2.0:1789760796.442714:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775508709332 time 775508448916 state 0x1 ref 1 00000100:00100000:2.0:1789760796.442788:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775508709332 time 775547497540 state 0x0 ref 1 00000100:00100000:2.0:1789760796.442803:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508493056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760796.442866:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25772032, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:2.0:1789760796.442876:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5731b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.442895:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760796.442901:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.442909:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5731b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.442927:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5731b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.481201:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973612 is committed 00002000:00010000:2.0:1789760796.481462:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25772032 -> 25776128 00010000:00010000:2.0:1789760796.481484:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5731b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.481500:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5731b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.481511:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5731b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.481520:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760796.481525:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5731b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.481537:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5731b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.481563:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5731b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.481572:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe5731b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25772032->25776127] (req 25772032->25776127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.481719:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508493056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38913us (39588us total) trans 4294973612 rc 0/0 00000100:00100000:2.0:1789760796.481731:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6324 00000100:00100000:2.0:1789760796.481744:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775508709332 time 775547497540 state 0x2 ref 0 00000100:00100000:0.0:1789760796.485129:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760796.485236:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508493184 00000100:00100000:2.0:1789760796.485249:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508493184 00000100:00100000:2.0:1789760796.485410:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775508709332 time 775547497540 state 0x0 ref 1 00000100:00100000:2.0:1789760796.485429:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775547757956 time 775547497540 state 0x1 ref 1 00000100:00100000:1.0:1789760796.485543:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775547757956 time 775590249634 state 0x0 ref 1 00000100:00100000:1.0:1789760796.485560:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508493184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760796.485760:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25776128, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0xc807 00010000:00010000:1.0:1789760796.485773:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57322 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.485795:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.485800:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.485808:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57322 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.485826:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57322 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.529367:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973613 is committed 00002000:00010000:0.0:1789760796.529640:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25776128 -> 25780224 00002000:00010000:0.0:1789760796.529646:0:8110:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760796 -> 1789760797 00002000:00010000:0.0:1789760796.529648:0:8110:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760796 -> 1789760797 00010000:00010000:0.0:1789760796.529675:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57322 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.529693:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57322 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.529703:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57322 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.529712:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760796.529718:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57322 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.529734:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57322 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.529762:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57322 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.529771:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57322 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25776128->25780223] (req 25776128->25780223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.530069:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508493184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44449us (44890us total) trans 4294973613 rc 0/0 00000100:00100000:0.0:1789760796.530083:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6325 00000100:00100000:0.0:1789760796.530096:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775547757956 time 775590249634 state 0x2 ref 0 00000100:00100000:3.0:1789760796.533235:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760796.533470:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508493312 00000100:00100000:0.0:1789760796.533484:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508493312 00000100:00100000:0.0:1789760796.533552:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775547757956 time 775590249634 state 0x0 ref 1 00000100:00100000:0.0:1789760796.533570:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775590510050 time 775590249634 state 0x1 ref 1 00000100:00100000:0.0:1789760796.533610:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775590510050 time 775638320269 state 0x0 ref 1 00000100:00100000:0.0:1789760796.533622:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508493312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760796.533692:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25780224, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:0.0:1789760796.533704:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57329 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.533724:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760796.533729:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.533736:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57329 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.533752:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57329 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.573239:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973614 is committed 00002000:00010000:1.0:1789760796.573653:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25780224 -> 25784320 00010000:00010000:1.0:1789760796.573684:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57329 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.573701:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57329 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.573711:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57329 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.573720:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760796.573726:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57329 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.573736:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57329 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.573764:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57329 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.573773:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57329 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25780224->25784319] (req 25780224->25784319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.574744:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508493312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41109us (41510us total) trans 4294973614 rc 0/0 00000100:00100000:1.0:1789760796.574760:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6326 00000100:00100000:1.0:1789760796.574772:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775590510050 time 775638320269 state 0x2 ref 0 00000100:00100000:0.0:1789760796.578185:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760796.578412:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508493440 00000100:00100000:2.0:1789760796.578425:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508493440 00000100:00100000:2.0:1789760796.578498:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775590510050 time 775638320269 state 0x0 ref 1 00000100:00100000:2.0:1789760796.578517:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775638580685 time 775638320269 state 0x1 ref 1 00000100:00100000:2.0:1789760796.578600:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775638580685 time 775683308312 state 0x0 ref 1 00000100:00100000:2.0:1789760796.578615:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508493440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760796.578690:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25784320, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:2.0:1789760796.578702:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe57330 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.578722:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:2.0:1789760796.578727:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.578736:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe57330 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.578754:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe57330 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.626324:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973615 is committed 00002000:00010000:3.0:1789760796.626809:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25784320 -> 25788416 00010000:00010000:3.0:1789760796.626831:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe57330 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.626847:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe57330 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.626927:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe57330 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.626936:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760796.626942:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe57330 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.626953:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe57330 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.626978:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe57330 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.626987:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe57330 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25784320->25788415] (req 25784320->25788415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.627544:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508493440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48923us (49365us total) trans 4294973615 rc 0/0 00000100:00100000:3.0:1789760796.627559:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6327 00000100:00100000:3.0:1789760796.627571:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775638580685 time 775683308312 state 0x2 ref 0 00000100:00100000:3.0:1789760796.631278:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760796.631428:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508493568 00000100:00100000:1.0:1789760796.631442:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508493568 00000100:00100000:1.0:1789760796.631528:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775638580685 time 775683308312 state 0x0 ref 1 00000100:00100000:1.0:1789760796.631548:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775683568728 time 775683308312 state 0x1 ref 1 00000100:00100000:1.0:1789760796.631594:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775683568728 time 775736303727 state 0x0 ref 1 00000100:00100000:1.0:1789760796.631608:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508493568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760796.631686:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25788416, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:1.0:1789760796.631696:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57337 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.631716:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.631722:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.631730:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57337 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.631748:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57337 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.670927:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973616 is committed 00002000:00010000:0.0:1789760796.671145:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25788416 -> 25792512 00010000:00010000:0.0:1789760796.671165:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57337 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.671179:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57337 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.671190:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57337 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.671198:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760796.671203:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57337 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.671214:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57337 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.671244:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57337 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.671251:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57337 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25788416->25792511] (req 25788416->25792511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.671420:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508493568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 39807us (40151us total) trans 4294973616 rc 0/0 00000100:00100000:0.0:1789760796.671431:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6328 00000100:00100000:0.0:1789760796.671444:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775683568728 time 775736303727 state 0x2 ref 0 00000100:00100000:2.0:1789760796.675720:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760796.675817:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508493696 00000100:00100000:1.0:1789760796.675830:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508493696 00000100:00100000:1.0:1789760796.675906:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775683568728 time 775736303727 state 0x0 ref 1 00000100:00100000:1.0:1789760796.675927:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775736564143 time 775736303727 state 0x1 ref 1 00000100:00100000:1.0:1789760796.675961:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775736564143 time 775780671268 state 0x0 ref 1 00000100:00100000:1.0:1789760796.675975:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508493696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760796.676051:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25792512, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:1.0:1789760796.676062:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5733e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.676081:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.676086:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.676093:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5733e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.676110:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5733e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.707321:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973617 is committed 00002000:00010000:0.0:1789760796.707453:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25792512 -> 25796608 00010000:00010000:0.0:1789760796.707474:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5733e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.707490:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5733e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.707500:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5733e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.707508:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760796.707515:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5733e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.707531:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5733e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.707558:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5733e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.707566:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe5733e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25792512->25796607] (req 25792512->25796607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.707810:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508493696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31829us (32098us total) trans 4294973617 rc 0/0 00000100:00100000:0.0:1789760796.707823:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6329 00000100:00100000:0.0:1789760796.707835:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775736564143 time 775780671268 state 0x2 ref 0 00000100:00100000:2.0:1789760796.711173:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760796.711433:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508493824 00000100:00100000:3.0:1789760796.711448:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508493824 00000100:00100000:3.0:1789760796.711530:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775736564143 time 775780671268 state 0x0 ref 1 00000100:00100000:3.0:1789760796.711554:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775780931684 time 775780671268 state 0x1 ref 1 00000100:00100000:3.0:1789760796.711632:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775780931684 time 775816340880 state 0x0 ref 1 00000100:00100000:3.0:1789760796.711650:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508493824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760796.711725:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25796608, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:3.0:1789760796.711737:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57345 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.711756:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:3.0:1789760796.711762:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.711769:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57345 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.711786:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57345 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.741828:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760796.742032:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508494080 00000100:00100000:1.0:1789760796.742047:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508494080 00000100:00100000:1.0:1789760796.742130:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 999 00000100:00100000:1.0:1789760796.742143:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508494080:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:1.0:1789760796.743704:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508494080:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 1550us (1887us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760796.743720:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 999 00000100:00100000:2.0:1789760796.744173:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760796.744760:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760796.744793:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760796.744881:0:6461:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508493952 00000100:00100000:3.0:1789760796.744894:0:6461:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508493952 00000100:00100000:3.0:1789760796.744955:0:6461:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 493 00000100:00100000:3.0:1789760796.744967:0:6461:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988caac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2409:x1876699508493952:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:3.0:1789760796.745155:0:6461:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988caac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2409:x1876699508493952:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 189us (983us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760796.745164:0:6461:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 493 00000100:00100000:2.0:1789760796.745357:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00080000:3.0:1789760796.745417:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760796.746743:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508494336 00000100:00100000:0.0:1789760796.746757:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508494336 00000100:00100000:0.0:1789760796.747105:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508494208 00000100:00100000:0.0:1789760796.747110:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508494208 00000100:00100000:0.0:1789760796.747138:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508494464 00000100:00100000:0.0:1789760796.747141:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508494464 00000100:00100000:0.0:1789760796.747170:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 1000 00000100:00100000:0.0:1789760796.747181:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508494336:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760796.747268:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508494336:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 89us (2510us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760796.747276:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 1000 00000100:00100000:0.0:1789760796.747312:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 1001 00000100:00100000:0.0:1789760796.747320:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508494208:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760796.747347:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508494208:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 27us (2553us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760796.747352:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 1001 00000100:00100000:0.0:1789760796.747364:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 1002 00000100:00100000:0.0:1789760796.747370:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508494464:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760796.747393:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508494464:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 23us (2037us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760796.747399:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 1002 00000100:00080000:3.0:1789760796.748210:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760796.748219:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789760796.758490:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973618 is committed 00002000:00010000:3.0:1789760796.758746:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25796608 -> 25800704 00010000:00010000:3.0:1789760796.758767:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57345 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.758783:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57345 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.758793:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57345 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.758803:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760796.758809:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57345 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.758820:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57345 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.758848:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57345 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.758857:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57345 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25796608->25800703] (req 25796608->25800703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.759101:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508493824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 47447us (47935us total) trans 4294973618 rc 0/0 00000100:00100000:3.0:1789760796.759115:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6330 00000100:00100000:3.0:1789760796.759128:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775780931684 time 775816340880 state 0x2 ref 0 00000100:00100000:2.0:1789760796.761844:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760796.761973:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508494592 00000100:00100000:0.0:1789760796.761987:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508494592 00000100:00100000:0.0:1789760796.762166:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775780931684 time 775816340880 state 0x0 ref 1 00000100:00100000:0.0:1789760796.762185:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775816601296 time 775816340880 state 0x1 ref 1 00000100:00100000:0.0:1789760796.762247:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775816601296 time 775866956928 state 0x0 ref 1 00000100:00100000:0.0:1789760796.762262:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508494592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760796.762356:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25800704, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:0.0:1789760796.762366:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5734c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.762385:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760796.762391:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.762399:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5734c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.762416:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5734c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.801997:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973619 is committed 00002000:00010000:0.0:1789760796.802243:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25800704 -> 25804800 00010000:00010000:0.0:1789760796.802264:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5734c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.802280:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5734c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.802931:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5734c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.802941:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760796.802948:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5734c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.802959:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5734c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.802985:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5734c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.802995:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5734c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25800704->25804799] (req 25800704->25804799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.803420:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508494592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41151us (41583us total) trans 4294973619 rc 0/0 00000100:00100000:0.0:1789760796.803434:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6331 00000100:00100000:0.0:1789760796.803447:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775816601296 time 775866956928 state 0x2 ref 0 00000100:00100000:2.0:1789760796.807888:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760796.807980:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508494720 00000100:00100000:1.0:1789760796.807993:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508494720 00000100:00100000:1.0:1789760796.808148:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775816601296 time 775866956928 state 0x0 ref 1 00000100:00100000:1.0:1789760796.808167:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775867217344 time 775866956928 state 0x1 ref 1 00000100:00100000:1.0:1789760796.808203:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775867217344 time 775912912793 state 0x0 ref 1 00000100:00100000:1.0:1789760796.808218:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508494720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760796.808313:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25804800, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:1.0:1789760796.808326:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57353 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.808349:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.808354:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.808363:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57353 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.808382:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57353 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760796.854728:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25804800 -> 25808896 00010000:00010000:0.0:1789760796.854758:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57353 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.854774:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57353 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.854784:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57353 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.854791:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760796.854796:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57353 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.854806:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57353 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.854833:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57353 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.854848:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57353 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25804800->25808895] (req 25804800->25808895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.854974:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508494720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46753us (47095us total) trans 4294973620 rc 0/0 00000100:00100000:0.0:1789760796.854988:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6332 00000100:00100000:0.0:1789760796.855001:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775867217344 time 775912912793 state 0x2 ref 0 00000001:00080000:3.0:1789760796.857194:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973620 is committed 00000100:00100000:3.0:1789760796.858142:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760796.858239:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508494848 00000100:00100000:0.0:1789760796.858253:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508494848 00000100:00100000:0.0:1789760796.858357:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775867217344 time 775912912793 state 0x0 ref 1 00000100:00100000:0.0:1789760796.858378:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775913173209 time 775912912793 state 0x1 ref 1 00000100:00100000:0.0:1789760796.858404:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775913173209 time 775963114498 state 0x0 ref 1 00000100:00100000:0.0:1789760796.858418:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508494848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760796.858483:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25808896, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:0.0:1789760796.858493:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5735a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.858513:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:0.0:1789760796.858519:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.858527:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5735a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.858549:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5735a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.889054:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973621 is committed 00002000:00010000:1.0:1789760796.889316:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25808896 -> 25812992 00010000:00010000:1.0:1789760796.889337:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5735a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.889365:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5735a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.889376:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5735a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.889384:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760796.889390:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5735a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.889401:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5735a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.889427:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5735a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.889436:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5735a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25808896->25812991] (req 25808896->25812991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.889569:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508494848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31145us (31437us total) trans 4294973621 rc 0/0 00000100:00100000:1.0:1789760796.889585:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6333 00000100:00100000:1.0:1789760796.889599:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775913173209 time 775963114498 state 0x2 ref 0 00000100:00100000:0.0:1789760796.893683:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760796.893787:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508494976 00000100:00100000:1.0:1789760796.893800:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508494976 00000100:00100000:1.0:1789760796.893872:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775913173209 time 775963114498 state 0x0 ref 1 00000100:00100000:1.0:1789760796.893891:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775963374914 time 775963114498 state 0x1 ref 1 00000100:00100000:1.0:1789760796.893972:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775963374914 time 775998680583 state 0x0 ref 1 00000100:00100000:1.0:1789760796.893987:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e61c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508494976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760796.894069:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25812992, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:1.0:1789760796.894079:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57361 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.894099:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=775s v=(0 0) 00010000:00010000:1.0:1789760796.894104:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.894111:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57361 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.894128:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57361 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760796.930991:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25812992 -> 25817088 00010000:00010000:0.0:1789760796.931029:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57361 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.931044:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57361 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.931054:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57361 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.931063:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760796.931069:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57361 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.931079:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57361 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.931106:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57361 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.931114:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe57361 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25812992->25817087] (req 25812992->25817087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.931239:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e61c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508494976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37248us (37572us total) trans 4294973622 rc 0/0 00000100:00100000:0.0:1789760796.931251:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6334 00000100:00100000:0.0:1789760796.931263:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775963374914 time 775998680583 state 0x2 ref 0 00000001:00080000:2.0:1789760796.933892:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973622 is committed 00000100:00100000:2.0:1789760796.934744:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760796.934841:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508495104 00000100:00100000:0.0:1789760796.934855:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508495104 00000100:00100000:0.0:1789760796.934929:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775963374914 time 775998680583 state 0x0 ref 1 00000100:00100000:0.0:1789760796.934949:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775998940999 time 775998680583 state 0x1 ref 1 00000100:00100000:0.0:1789760796.935023:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775998940999 time 776039731705 state 0x0 ref 1 00000100:00100000:0.0:1789760796.935038:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508495104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760796.935100:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25817088, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:0.0:1789760796.935111:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57368 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.935131:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760796.935136:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.935144:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57368 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.935159:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57368 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760796.976609:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25817088 -> 25821184 00010000:00010000:1.0:1789760796.976639:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57368 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.976656:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57368 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.976677:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57368 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.976685:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760796.976691:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57368 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.976702:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57368 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.976728:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57368 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.976735:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57368 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25817088->25821183] (req 25817088->25821183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.976864:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508495104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 41821us (42121us total) trans 4294973623 rc 0/0 00000100:00100000:1.0:1789760796.976878:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6335 00000100:00100000:1.0:1789760796.976892:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775998940999 time 776039731705 state 0x2 ref 0 00000001:00080000:3.0:1789760796.979682:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973623 is committed 00000100:00100000:1.0:1789760796.979847:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760796.979948:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508495232 00000100:00100000:2.0:1789760796.979962:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508495232 00000100:00100000:2.0:1789760796.980046:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 775998940999 time 776039731705 state 0x0 ref 1 00000100:00100000:2.0:1789760796.980064:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776039992121 time 776039731705 state 0x1 ref 1 00000100:00100000:2.0:1789760796.980135:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776039992121 time 776084843740 state 0x0 ref 1 00000100:00100000:2.0:1789760796.980148:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508495232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760796.980212:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25821184, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:2.0:1789760796.980223:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5736f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.980243:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760796.980248:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.980257:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5736f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.980274:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5736f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760797.014719:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25821184 -> 25825280 00010000:00010000:3.0:1789760797.014749:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5736f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014767:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5736f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014777:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5736f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014785:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760797.014792:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5736f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014802:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5736f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014830:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5736f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014838:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000044165b1c/0xb49177c0efe5736f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25821184->25825279] (req 25821184->25825279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.014968:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508495232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 34814us (35120us total) trans 4294973624 rc 0/0 00000100:00100000:3.0:1789760797.014982:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6336 00000100:00100000:3.0:1789760797.014994:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776039992121 time 776084843740 state 0x2 ref 0 00000001:00080000:3.0:1789760797.015058:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973624 is committed 00000100:00100000:2.0:1789760797.018930:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760797.019040:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508495360 00000100:00100000:0.0:1789760797.019056:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508495360 00000100:00100000:0.0:1789760797.019151:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776039992121 time 776084843740 state 0x0 ref 1 00000100:00100000:0.0:1789760797.019171:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776085104156 time 776084843740 state 0x1 ref 1 00000100:00100000:0.0:1789760797.019212:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776085104156 time 776123921791 state 0x0 ref 1 00000100:00100000:0.0:1789760797.019225:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508495360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760797.019330:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25825280, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:0.0:1789760797.019342:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe57376 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.019362:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760797.019368:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.019375:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe57376 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.019394:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe57376 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760797.020244:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760797.057452:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973625 is committed 00002000:00010000:3.0:1789760797.057801:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25825280 -> 25829376 00010000:00010000:3.0:1789760797.057821:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe57376 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.057833:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe57376 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.057839:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe57376 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.057896:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760797.057901:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe57376 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.057909:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe57376 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.057933:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe57376 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.057939:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ce824f6/0xb49177c0efe57376 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25825280->25829375] (req 25825280->25829375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.058049:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508495360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38819us (39130us total) trans 4294973625 rc 0/0 00000100:00100000:3.0:1789760797.058062:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6337 00000100:00100000:3.0:1789760797.058074:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776085104156 time 776123921791 state 0x2 ref 0 00000100:00100000:2.0:1789760797.060494:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760797.060612:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508495488 00000100:00100000:1.0:1789760797.060627:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508495488 00000100:00100000:1.0:1789760797.060707:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776085104156 time 776123921791 state 0x0 ref 1 00000100:00100000:1.0:1789760797.060725:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776124182207 time 776123921791 state 0x1 ref 1 00000100:00100000:1.0:1789760797.060763:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776124182207 time 776165473432 state 0x0 ref 1 00000100:00100000:1.0:1789760797.060777:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508495488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760797.060842:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25829376, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:1.0:1789760797.060853:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5737d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.060873:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.060879:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.060888:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5737d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.060908:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5737d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.110827:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973626 is committed 00002000:00010000:3.0:1789760797.111258:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25829376 -> 25833472 00010000:00010000:3.0:1789760797.111281:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5737d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.111337:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5737d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.111353:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5737d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.111361:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760797.111366:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5737d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.111376:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5737d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.111403:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5737d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.111412:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5737d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25829376->25833471] (req 25829376->25833471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.111600:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508495488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50818us (51108us total) trans 4294973626 rc 0/0 00000100:00100000:3.0:1789760797.111609:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6338 00000100:00100000:3.0:1789760797.111620:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776124182207 time 776165473432 state 0x2 ref 0 00000100:00100000:1.0:1789760797.116833:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760797.117126:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508495616 00000100:00100000:3.0:1789760797.117140:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508495616 00000100:00100000:3.0:1789760797.117210:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776124182207 time 776165473432 state 0x0 ref 1 00000100:00100000:3.0:1789760797.117225:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776165733848 time 776165473432 state 0x1 ref 1 00000100:00100000:3.0:1789760797.117255:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776165733848 time 776221964744 state 0x0 ref 1 00000100:00100000:3.0:1789760797.117268:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508495616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760797.117437:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25833472, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:3.0:1789760797.117452:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57384 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.117472:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760797.117477:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.117484:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57384 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.117501:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57384 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.171550:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973627 is committed 00002000:00010000:2.0:1789760797.171933:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25833472 -> 25837568 00010000:00010000:2.0:1789760797.171955:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57384 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.171970:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57384 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.171981:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57384 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.171989:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760797.171994:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57384 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.172004:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57384 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.172031:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57384 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.172046:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57384 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25833472->25837567] (req 25833472->25837567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.172220:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508495616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 54945us (55394us total) trans 4294973627 rc 0/0 00000100:00100000:2.0:1789760797.172232:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6339 00000100:00100000:2.0:1789760797.172243:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776165733848 time 776221964744 state 0x2 ref 0 00000100:00100000:2.0:1789760797.175412:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760797.175923:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508495744 00000100:00100000:3.0:1789760797.175938:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508495744 00000100:00100000:3.0:1789760797.176021:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776165733848 time 776221964744 state 0x0 ref 1 00000100:00100000:3.0:1789760797.176041:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776222225160 time 776221964744 state 0x1 ref 1 00000100:00100000:3.0:1789760797.176074:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776222225160 time 776280784861 state 0x0 ref 1 00000100:00100000:3.0:1789760797.176087:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508495744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760797.176154:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25837568, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:3.0:1789760797.176166:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5738b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.176187:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760797.176191:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.176198:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5738b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.176215:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5738b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.225821:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973628 is committed 00002000:00010000:3.0:1789760797.225916:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25837568 -> 25841664 00010000:00010000:3.0:1789760797.225935:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5738b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.225952:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5738b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.225961:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5738b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.225971:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760797.225976:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5738b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.225986:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5738b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.226020:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5738b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.226028:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5738b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25837568->25841663] (req 25837568->25841663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.226145:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508495744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 50053us (50742us total) trans 4294973628 rc 0/0 00000100:00100000:3.0:1789760797.226157:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6340 00000100:00100000:3.0:1789760797.226171:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776222225160 time 776280784861 state 0x2 ref 0 00000100:00100000:3.0:1789760797.230117:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760797.230502:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508495872 00000100:00100000:1.0:1789760797.230515:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508495872 00000100:00100000:1.0:1789760797.230584:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776222225160 time 776280784861 state 0x0 ref 1 00000100:00100000:1.0:1789760797.230604:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776281045277 time 776280784861 state 0x1 ref 1 00000100:00100000:1.0:1789760797.230696:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776281045277 time 776335405918 state 0x0 ref 1 00000100:00100000:1.0:1789760797.230711:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508495872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760797.230777:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25841664, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:1.0:1789760797.230788:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57392 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.230808:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.230813:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.230820:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57392 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.230836:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57392 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.312043:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973629 is committed 00002000:00010000:3.0:1789760797.312239:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25841664 -> 25845760 00010000:00010000:3.0:1789760797.312258:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57392 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.312273:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57392 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.312323:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57392 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.312333:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760797.312339:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57392 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.312354:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57392 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.312381:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57392 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.312389:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57392 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25841664->25845759] (req 25841664->25845759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.312517:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508495872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 81800us (82408us total) trans 4294973629 rc 0/0 00000100:00100000:3.0:1789760797.312529:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6341 00000100:00100000:3.0:1789760797.312541:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776281045277 time 776335405918 state 0x2 ref 0 00000100:00100000:1.0:1789760797.319522:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760797.319592:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508496000 00000100:00100000:2.0:1789760797.319605:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508496000 00000100:00100000:2.0:1789760797.319701:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776281045277 time 776335405918 state 0x0 ref 1 00000100:00100000:2.0:1789760797.319720:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776335666334 time 776335405918 state 0x1 ref 1 00000100:00100000:2.0:1789760797.319754:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776335666334 time 776424463216 state 0x0 ref 1 00000100:00100000:2.0:1789760797.319769:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508496000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760797.319906:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25845760, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:2.0:1789760797.319918:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57399 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.319941:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760797.319947:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.319956:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57399 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.319972:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57399 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.359946:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973630 is committed 00002000:00010000:0.0:1789760797.360173:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25845760 -> 25849856 00010000:00010000:0.0:1789760797.360193:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57399 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.360209:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57399 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.360218:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57399 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.360225:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760797.360231:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57399 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.360241:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57399 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.360266:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57399 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.360275:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db104144/0xb49177c0efe57399 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25845760->25849855] (req 25845760->25849855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.360750:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508496000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40977us (41234us total) trans 4294973630 rc 0/0 00000100:00100000:0.0:1789760797.360763:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6342 00000100:00100000:0.0:1789760797.360775:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776335666334 time 776424463216 state 0x2 ref 0 00000100:00100000:2.0:1789760797.369034:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760797.369158:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508496128 00000100:00100000:3.0:1789760797.369172:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508496128 00000100:00100000:3.0:1789760797.369256:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776335666334 time 776424463216 state 0x0 ref 1 00000100:00100000:3.0:1789760797.369277:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776424723632 time 776424463216 state 0x1 ref 1 00000100:00100000:3.0:1789760797.369357:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776424723632 time 776474066252 state 0x0 ref 1 00000100:00100000:3.0:1789760797.369372:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508496128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760797.369437:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25849856, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:3.0:1789760797.369449:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.369470:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760797.369475:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.369483:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.369500:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.415339:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973631 is committed 00002000:00010000:3.0:1789760797.415669:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25849856 -> 25853952 00010000:00010000:3.0:1789760797.415690:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.415706:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.415716:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.415724:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760797.415730:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.415739:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.415763:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.415773:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25849856->25853951] (req 25849856->25853951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.415899:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508496128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46522us (46874us total) trans 4294973631 rc 0/0 00000100:00100000:3.0:1789760797.415911:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6343 00000100:00100000:3.0:1789760797.415923:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776424723632 time 776474066252 state 0x2 ref 0 00000100:00100000:2.0:1789760797.420376:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760797.420778:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508496256 00000100:00100000:3.0:1789760797.420793:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508496256 00000100:00100000:3.0:1789760797.420874:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776424723632 time 776474066252 state 0x0 ref 1 00000100:00100000:3.0:1789760797.420895:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776474326668 time 776474066252 state 0x1 ref 1 00000100:00100000:3.0:1789760797.420931:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776474326668 time 776525641483 state 0x0 ref 1 00000100:00100000:3.0:1789760797.420946:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508496256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760797.421025:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25853952, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:3.0:1789760797.421036:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe573a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.421055:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760797.421061:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.421069:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe573a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.421091:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe573a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.443247:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760797.478149:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973632 is committed 00002000:00010000:2.0:1789760797.478382:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25853952 -> 25858048 00010000:00010000:2.0:1789760797.478406:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe573a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.478421:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe573a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.478431:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe573a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.478439:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760797.478444:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe573a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.478457:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe573a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.478482:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe573a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.478492:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe573a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25853952->25858047] (req 25853952->25858047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.478853:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508496256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 57902us (58487us total) trans 4294973632 rc 0/0 00000100:00100000:2.0:1789760797.478867:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6344 00000100:00100000:2.0:1789760797.478881:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776474326668 time 776525641483 state 0x2 ref 0 00000100:00100000:1.0:1789760797.484562:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760797.484796:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508496384 00000100:00100000:0.0:1789760797.484811:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508496384 00000100:00100000:0.0:1789760797.484884:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776474326668 time 776525641483 state 0x0 ref 1 00000100:00100000:0.0:1789760797.484903:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776525901899 time 776525641483 state 0x1 ref 1 00000100:00100000:0.0:1789760797.484990:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776525901899 time 776589699230 state 0x0 ref 1 00000100:00100000:0.0:1789760797.485004:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508496384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760797.485066:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25858048, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0xc807 00010000:00010000:0.0:1789760797.485078:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe573ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.485098:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760797.485103:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.485111:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe573ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.485127:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe573ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760797.545544:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25858048 -> 25862144 00002000:00010000:2.0:1789760797.545556:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760797 -> 1789760798 00002000:00010000:2.0:1789760797.545559:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760797 -> 1789760798 00010000:00010000:2.0:1789760797.545587:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe573ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.545603:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe573ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.545619:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe573ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.545627:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760797.545632:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe573ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.545642:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe573ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.545676:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe573ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.545685:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d683aef/0xb49177c0efe573ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25858048->25862143] (req 25858048->25862143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.545802:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508496384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 60793us (61250us total) trans 4294973633 rc 0/0 00000100:00100000:2.0:1789760797.545815:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6345 00000100:00100000:2.0:1789760797.545827:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776525901899 time 776589699230 state 0x2 ref 0 00000001:00080000:1.0:1789760797.546263:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973633 is committed 00000100:00100000:3.0:1789760797.549379:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760797.549565:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508496512 00000100:00100000:1.0:1789760797.549578:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508496512 00000100:00100000:1.0:1789760797.549749:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776525901899 time 776589699230 state 0x0 ref 1 00000100:00100000:1.0:1789760797.549769:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776589959646 time 776589699230 state 0x1 ref 1 00000100:00100000:1.0:1789760797.549800:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776589959646 time 776654510134 state 0x0 ref 1 00000100:00100000:1.0:1789760797.549814:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508496512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760797.549880:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25862144, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:1.0:1789760797.549891:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.549912:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.549917:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.549925:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.549942:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.566913:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04887e61880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534093824:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760797.567035:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.567136:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534093824 00000100:00100000:1.0:1789760797.567145:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534093824 00000100:00080000:0.0:1789760797.567218:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760797.567231:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:1.0:1789760797.567244:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534093824:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760797.567318:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760797.567367:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534093824:12345-0@lo:13:osp-pre-1-0.0 Request processed in 126us (333us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.567376:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:0.0:1789760797.567517:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04887e61880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534093824:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760797.568019:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04888047b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534093952:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760797.568063:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.568346:0:8106:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534093952 00000100:00100000:1.0:1789760797.568353:0:8106:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534093952 00000100:00100000:1.0:1789760797.568509:0:8106:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:1.0:1789760797.568520:0:8106:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be419c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534093952:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760797.568543:0:8106:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3832545280 avail 3818913792 00000100:00100000:1.0:1789760797.568590:0:8106:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be419c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534093952:12345-0@lo:13:osp-pre-0-1.0 Request processed in 72us (525us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.568597:0:8106:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:1.0:1789760797.570076:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04985469880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534094464:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.570140:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.570187:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3472:1876699534094976:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.570214:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.570235:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534095488:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.570259:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.570344:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04888047b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534093952:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760797.570368:0:3472:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:1.0:1789760797.570379:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa04985468380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534094080:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760797.570415:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.570440:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049be26f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534094592:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.570462:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.570484:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534095104:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.570509:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.570530:0:3472:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049beaf9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534095616:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760797.570550:0:3472:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760797.570589:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534094464 00000100:00100000:1.0:1789760797.570596:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534094464 00000100:00100000:1.0:1789760797.570645:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:1.0:1789760797.570666:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534094464:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.570725:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3472:x1876699534094464:12345-0@lo:400:kworker.0 Request processed in 70us (583us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.570734:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00100000:1.0:1789760797.570763:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534094976 00000100:00100000:1.0:1789760797.570767:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534094976 00000100:00100000:1.0:1789760797.570799:0:6468:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534094592 00000100:00100000:1.0:1789760797.570802:0:6468:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534094592 00000100:00100000:1.0:1789760797.570823:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1003 00000100:00100000:1.0:1789760797.570830:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3472:x1876699534094976:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.570862:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3472:x1876699534094976:12345-0@lo:400:kworker.0 Request processed in 32us (646us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.570868:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1003 00000100:00100000:1.0:1789760797.570882:0:6468:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1004 00000100:00100000:1.0:1789760797.570888:0:6468:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534094592:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.570916:0:6468:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3472:x1876699534094592:12345-0@lo:400:kworker.0 Request processed in 28us (453us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.570922:0:6468:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1004 00000100:00100000:1.0:1789760797.570947:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534095488 00000100:00100000:1.0:1789760797.570951:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534095488 00000100:00100000:1.0:1789760797.570986:0:8102:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534095104 00000100:00100000:1.0:1789760797.570989:0:8102:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534095104 00000100:00100000:1.0:1789760797.571012:0:8102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 256 00000100:00100000:1.0:1789760797.571019:0:8102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534095488:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.571059:0:8102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534095488:12345-0@lo:400:kworker.0 Request processed in 40us (798us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.571067:0:8102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 256 00000100:00100000:1.0:1789760797.571083:0:8102:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 257 00000100:00100000:1.0:1789760797.571091:0:8102:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534095104:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.571110:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0498546b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3474:1876699534094208:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.571121:0:8102:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534095104:12345-0@lo:400:kworker.0 Request processed in 30us (610us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.571127:0:8102:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 257 00000100:00100000:1.0:1789760797.571172:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534094080 00000100:00100000:1.0:1789760797.571175:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534094080 00000100:00100000:1.0:1789760797.571194:0:8107:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534095616 00000100:00100000:1.0:1789760797.571197:0:8107:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534095616 00000100:00100000:1.0:1789760797.571207:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:1.0:1789760797.571213:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534094080:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760797.571226:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40697856 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789760797.571738:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3472:x1876699534094080:12345-0@lo:13:osp-pre-1-1.0 Request processed in 522us (1319us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.571747:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:1.0:1789760797.571763:0:8107:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 415 00000100:00100000:1.0:1789760797.571770:0:8107:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd99f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534095616:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760797.571783:0:8107:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 527761408 pending 1703936 free 3832545280 avail 3818913792 00000100:00100000:1.0:1789760797.571813:0:8107:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd99f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3472:x1876699534095616:12345-0@lo:13:osp-pre-0-0.0 Request processed in 44us (1262us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.571819:0:8107:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1789760797.572034:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760797.572080:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049be26f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534094720:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.572114:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760797.572133:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534095232:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.572281:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760797.572341:0:3474:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:1.0:1789760797.572343:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04985468380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534094080:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760797.572352:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa0498546a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534094336:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.572362:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049be26f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3472:1876699534094592:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.572377:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048846d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534095104:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.572385:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760797.572408:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534094848:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.572434:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760797.572453:0:3474:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534095360:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.572476:0:3474:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760797.572507:0:6460:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534094208 00000100:00100000:1.0:1789760797.572512:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049beaf9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3472:1876699534095616:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760797.572514:0:6460:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534094208 00000100:00100000:1.0:1789760797.572537:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa04985469880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534094464:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.572554:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048846d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3472:1876699534094976:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.572560:0:6460:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 494 00000100:00100000:1.0:1789760797.572567:0:3472:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048846d3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3472:1876699534095488:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.572573:0:6460:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988ca9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3474:x1876699534094208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.572621:0:6460:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988ca9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:b334fb96-4b5c-4ca1-8c00-a1c62fe37378+24:3474:x1876699534094208:12345-0@lo:400:kworker.0 Request processed in 53us (588us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760797.572628:0:6460:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 494 00000100:00100000:2.0:1789760797.573080:0:6471:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534094720 00000100:00100000:2.0:1789760797.573086:0:6471:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534094720 00000100:00100000:2.0:1789760797.573119:0:6471:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 475 00000100:00100000:2.0:1789760797.573125:0:6471:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534094720:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.573246:0:6471:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3474:x1876699534094720:12345-0@lo:400:kworker.0 Request processed in 120us (1130us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760797.573253:0:6471:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 475 00000100:00100000:2.0:1789760797.573278:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534095232 00000100:00100000:2.0:1789760797.573282:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534095232 00000100:00100000:1.0:1789760797.573331:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534094336 00000100:00100000:1.0:1789760797.573337:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534094336 00000100:00100000:2.0:1789760797.573342:0:6466:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534094848 00000100:00100000:2.0:1789760797.573347:0:6466:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534094848 00000100:00100000:1.0:1789760797.573371:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534095360 00000100:00100000:1.0:1789760797.573375:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534095360 00000100:00100000:2.0:1789760797.573435:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1005 00000100:00100000:2.0:1789760797.573444:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534095232:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.573480:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1006 00000100:00100000:2.0:1789760797.573486:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3474:x1876699534095232:12345-0@lo:400:kworker.0 Request processed in 43us (1206us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.573493:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534094336:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.573499:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1005 00000100:00100000:2.0:1789760797.573517:0:6466:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1007 00000100:00100000:1.0:1789760797.573540:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3474:x1876699534094336:12345-0@lo:400:kworker.0 Request processed in 51us (1155us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.573547:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1006 00000100:00100000:1.0:1789760797.573560:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1008 00000100:00100000:1.0:1789760797.573567:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888046680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534095360:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.573594:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888046680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3474:x1876699534095360:12345-0@lo:400:kworker.0 Request processed in 28us (1117us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.573601:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1008 00000100:00100000:2.0:1789760797.573648:0:6466:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534094848:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.573695:0:6466:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888047b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3474:x1876699534094848:12345-0@lo:400:kworker.0 Request processed in 171us (1259us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760797.573703:0:6466:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1007 00000100:00100000:2.0:1789760797.574031:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0498546a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3474:1876699534094336:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.574042:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048846d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3474:1876699534094848:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.574053:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048846d2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3474:1876699534095360:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.574063:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa0498546b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:b334fb96-4b5c-4ca1-8c00-a1c62fe37378:3474:1876699534094208:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.574074:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049be26f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3474:1876699534094720:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.574083:0:3474:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048846d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3474:1876699534095232:0@lo:400:kworker.0 00000100:00080000:2.0:1789760797.574121:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789760797.574128:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789760797.574132:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760797.574134:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760797.623695:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973634 is committed 00002000:00010000:1.0:1789760797.623812:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25862144 -> 25866240 00010000:00010000:1.0:1789760797.623833:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.623849:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.623859:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.623868:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760797.623874:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.623885:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.623911:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.623922:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25862144->25866239] (req 25862144->25866239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.624150:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508496512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 74332us (74780us total) trans 4294973634 rc 0/0 00000100:00100000:1.0:1789760797.624163:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6346 00000100:00100000:1.0:1789760797.624177:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776589959646 time 776654510134 state 0x2 ref 0 00000100:00100000:2.0:1789760797.629395:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760797.629500:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508496640 00000100:00100000:3.0:1789760797.629512:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508496640 00000100:00100000:3.0:1789760797.629723:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776589959646 time 776654510134 state 0x0 ref 1 00000100:00100000:3.0:1789760797.629743:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776654770550 time 776654510134 state 0x1 ref 1 00000100:00100000:3.0:1789760797.629771:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776654770550 time 776734480868 state 0x0 ref 1 00000100:00100000:3.0:1789760797.629786:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508496640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760797.629855:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25866240, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:3.0:1789760797.629865:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.629884:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:3.0:1789760797.629889:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.629896:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.629913:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.685332:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973635 is committed 00002000:00010000:3.0:1789760797.685490:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25866240 -> 25870336 00010000:00010000:3.0:1789760797.685511:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.685528:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.685539:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.685547:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760797.685553:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.685563:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.685589:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.685598:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25866240->25870335] (req 25866240->25870335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.685742:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04888044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508496640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 55950us (56354us total) trans 4294973635 rc 0/0 00000100:00100000:3.0:1789760797.685758:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6347 00000100:00100000:3.0:1789760797.685771:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776654770550 time 776734480868 state 0x2 ref 0 00000100:00100000:2.0:1789760797.691440:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760797.691663:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508496768 00000100:00100000:1.0:1789760797.691676:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508496768 00000100:00100000:1.0:1789760797.691748:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776654770550 time 776734480868 state 0x0 ref 1 00000100:00100000:1.0:1789760797.691767:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776734741284 time 776734480868 state 0x1 ref 1 00000100:00100000:2.0:1789760797.691865:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776734741284 time 776796572673 state 0x0 ref 1 00000100:00100000:2.0:1789760797.691882:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508496768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760797.691955:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25870336, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:2.0:1789760797.691968:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe573c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.691989:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:2.0:1789760797.691994:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.692012:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe573c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.692031:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe573c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.730358:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973636 is committed 00002000:00010000:1.0:1789760797.730448:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25870336 -> 25874432 00010000:00010000:1.0:1789760797.730468:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe573c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.730484:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe573c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.730494:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe573c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.730502:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760797.730508:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe573c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.730518:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe573c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.730543:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe573c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.730551:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe573c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25870336->25874431] (req 25870336->25874431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.730689:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508496768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 38794us (39247us total) trans 4294973636 rc 0/0 00000100:00100000:1.0:1789760797.730701:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6348 00000100:00100000:1.0:1789760797.730712:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776734741284 time 776796572673 state 0x2 ref 0 00000100:00100000:2.0:1789760797.736590:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760797.736697:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508496896 00000100:00100000:1.0:1789760797.736711:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508496896 00000100:00100000:1.0:1789760797.736786:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776734741284 time 776796572673 state 0x0 ref 1 00000100:00100000:1.0:1789760797.736806:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776796833089 time 776796572673 state 0x1 ref 1 00000100:00100000:1.0:1789760797.736843:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776796833089 time 776841552800 state 0x0 ref 1 00000100:00100000:1.0:1789760797.736857:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508496896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760797.736919:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25874432, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:1.0:1789760797.736930:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.736950:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.736955:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.736963:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.736980:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760797.785629:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25874432 -> 25878528 00010000:00010000:3.0:1789760797.785673:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.785690:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.785700:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.785708:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760797.785714:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.785724:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.785749:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.785922:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe573ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25874432->25878527] (req 25874432->25878527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.786066:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508496896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 49204us (49484us total) trans 4294973637 rc 0/0 00000100:00100000:3.0:1789760797.786079:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6349 00000100:00100000:3.0:1789760797.786209:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776796833089 time 776841552800 state 0x2 ref 0 00000001:00080000:1.0:1789760797.787729:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973637 is committed 00000100:00100000:0.0:1789760797.789525:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760797.789751:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508497024 00000100:00100000:3.0:1789760797.789825:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508497024 00000100:00100000:0.0:1789760797.789981:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776796833089 time 776841552800 state 0x0 ref 1 00000100:00100000:0.0:1789760797.790016:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776841813216 time 776841552800 state 0x1 ref 1 00000100:00100000:0.0:1789760797.790112:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776841813216 time 776894820837 state 0x0 ref 1 00000100:00100000:0.0:1789760797.790129:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508497024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760797.790203:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25878528, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:0.0:1789760797.790214:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe573d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.790235:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:0.0:1789760797.790241:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.790248:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe573d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.790265:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe573d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.838585:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973638 is committed 00002000:00010000:3.0:1789760797.838801:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25878528 -> 25882624 00010000:00010000:3.0:1789760797.838822:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe573d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.838839:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe573d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.838849:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe573d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.838857:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760797.838862:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe573d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.838872:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe573d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.838900:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe573d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.838915:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dff4988d/0xb49177c0efe573d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25878528->25882623] (req 25878528->25882623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.839081:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508497024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48948us (49793us total) trans 4294973638 rc 0/0 00000100:00100000:3.0:1789760797.839094:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6350 00000100:00100000:3.0:1789760797.839106:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776841813216 time 776894820837 state 0x2 ref 0 00000100:00100000:2.0:1789760797.844375:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760797.844520:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508497152 00000100:00100000:1.0:1789760797.844533:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508497152 00000100:00100000:1.0:1789760797.844616:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776841813216 time 776894820837 state 0x0 ref 1 00000100:00100000:1.0:1789760797.844640:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776895081253 time 776894820837 state 0x1 ref 1 00000100:00100000:1.0:1789760797.844673:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776895081253 time 776949382922 state 0x0 ref 1 00000100:00100000:1.0:1789760797.844688:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508497152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760797.844752:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25882624, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:1.0:1789760797.844763:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.844784:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=776s v=(0 0) 00010000:00010000:1.0:1789760797.844789:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.844797:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.844815:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.896398:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973639 is committed 00002000:00010000:1.0:1789760797.896520:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25882624 -> 25886720 00010000:00010000:1.0:1789760797.896541:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.896565:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.896576:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.896589:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760797.896594:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.896603:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.896630:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.896638:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25882624->25886719] (req 25882624->25886719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.896872:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508497152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 52179us (52503us total) trans 4294973639 rc 0/0 00000100:00100000:1.0:1789760797.896886:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6351 00000100:00100000:1.0:1789760797.896898:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776895081253 time 776949382922 state 0x2 ref 0 00000100:00100000:0.0:1789760797.901780:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760797.901855:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508497280 00000100:00100000:3.0:1789760797.901869:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508497280 00000100:00100000:3.0:1789760797.901947:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776895081253 time 776949382922 state 0x0 ref 1 00000100:00100000:3.0:1789760797.901967:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776949643338 time 776949382922 state 0x1 ref 1 00000100:00100000:3.0:1789760797.902003:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776949643338 time 777006712956 state 0x0 ref 1 00000100:00100000:3.0:1789760797.902026:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e60000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508497280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760797.902090:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25886720, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:3.0:1789760797.902101:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.902122:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760797.902128:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.902136:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.902159:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.954666:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973640 is committed 00002000:00010000:3.0:1789760797.954905:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25886720 -> 25890816 00010000:00010000:3.0:1789760797.954928:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.954944:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.954954:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.954963:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760797.954969:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.954993:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.955021:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.955030:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25886720->25890815] (req 25886720->25890815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.955160:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e60000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508497280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 53129us (53389us total) trans 4294973640 rc 0/0 00000100:00100000:3.0:1789760797.955172:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6352 00000100:00100000:3.0:1789760797.955185:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776949643338 time 777006712956 state 0x2 ref 0 00000100:00100000:0.0:1789760797.962008:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760797.962194:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508497408 00000100:00100000:3.0:1789760797.962208:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508497408 00000100:00100000:3.0:1789760797.962283:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 776949643338 time 777006712956 state 0x0 ref 1 00000100:00100000:3.0:1789760797.962325:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777006973372 time 777006712956 state 0x1 ref 1 00000100:00100000:3.0:1789760797.962359:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777006973372 time 777067069561 state 0x0 ref 1 00000100:00100000:3.0:1789760797.962373:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508497408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760797.962438:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25890816, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:3.0:1789760797.962449:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.962470:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760797.962475:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.962483:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.962500:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.026117:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973641 is committed 00002000:00010000:3.0:1789760798.026339:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25890816 -> 25894912 00010000:00010000:3.0:1789760798.026361:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.026377:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.026387:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.026396:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.026402:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.026413:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.026440:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.026450:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25890816->25894911] (req 25890816->25894911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.026705:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508497408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 64327us (64704us total) trans 4294973641 rc 0/0 00000100:00100000:3.0:1789760798.026724:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6353 00000100:00100000:3.0:1789760798.026737:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777006973372 time 777067069561 state 0x2 ref 0 00000100:00100000:2.0:1789760798.030208:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760798.030370:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508497536 00000100:00100000:1.0:1789760798.030384:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508497536 00000100:00100000:1.0:1789760798.030472:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777006973372 time 777067069561 state 0x0 ref 1 00000100:00100000:1.0:1789760798.030492:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777067329977 time 777067069561 state 0x1 ref 1 00000100:00100000:1.0:1789760798.030594:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777067329977 time 777135302897 state 0x0 ref 1 00000100:00100000:1.0:1789760798.030612:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508497536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760798.030685:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25894912, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:1.0:1789760798.030695:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.030714:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760798.030719:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.030726:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.030751:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.076898:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973642 is committed 00002000:00010000:1.0:1789760798.077054:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25894912 -> 25899008 00010000:00010000:1.0:1789760798.077072:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.077084:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.077091:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.077096:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760798.077100:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.077108:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.077129:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.077133:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe573ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25894912->25899007] (req 25894912->25899007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.077359:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508497536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46675us (47092us total) trans 4294973642 rc 0/0 00000100:00100000:1.0:1789760798.077372:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6354 00000100:00100000:1.0:1789760798.077386:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777067329977 time 777135302897 state 0x2 ref 0 00000100:00100000:0.0:1789760798.079481:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.079638:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508497664 00000100:00100000:3.0:1789760798.079662:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508497664 00000100:00100000:3.0:1789760798.079734:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777067329977 time 777135302897 state 0x0 ref 1 00000100:00100000:3.0:1789760798.079753:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777135563313 time 777135302897 state 0x1 ref 1 00000100:00100000:3.0:1789760798.079819:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777135563313 time 777184528384 state 0x0 ref 1 00000100:00100000:3.0:1789760798.079832:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c1fe850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508497664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.079896:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25899008, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:3.0:1789760798.079907:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.079927:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.079933:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.079941:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.079959:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.126317:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973643 is committed 00002000:00010000:3.0:1789760798.126347:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25899008 -> 25903104 00010000:00010000:3.0:1789760798.126368:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.126383:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.126394:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.126402:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.126408:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.126417:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.126441:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.126449:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe573f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25899008->25903103] (req 25899008->25903103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.126584:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c1fe850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508497664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 46747us (47108us total) trans 4294973643 rc 0/0 00000100:00100000:3.0:1789760798.126596:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6355 00000100:00100000:3.0:1789760798.126606:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777135563313 time 777184528384 state 0x2 ref 0 00000100:00100000:0.0:1789760798.130728:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.130803:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508497792 00000100:00100000:3.0:1789760798.130816:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508497792 00000100:00100000:3.0:1789760798.130900:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777135563313 time 777184528384 state 0x0 ref 1 00000100:00100000:3.0:1789760798.130921:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777184788800 time 777184528384 state 0x1 ref 1 00000100:00100000:3.0:1789760798.130954:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777184788800 time 777235664058 state 0x0 ref 1 00000100:00100000:3.0:1789760798.130969:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e62680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508497792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.131055:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25903104, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:3.0:1789760798.131066:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.131084:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.131089:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.131096:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.131113:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760798.164752:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25903104 -> 25907200 00010000:00010000:0.0:1789760798.164772:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.164786:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.164796:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.164804:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760798.164809:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.164819:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.164839:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.164847:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe573fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25903104->25907199] (req 25903104->25907199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.164950:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e62680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2409:x1876699508497792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33972us (34224us total) trans 4294973644 rc 0/0 00000100:00100000:0.0:1789760798.164961:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6356 00000100:00100000:0.0:1789760798.164971:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777184788800 time 777235664058 state 0x2 ref 0 00000001:00080000:1.0:1789760798.167313:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973644 is committed 00000100:00100000:2.0:1789760798.167702:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760798.167823:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508497920 00000100:00100000:1.0:1789760798.167832:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508497920 00000100:00100000:1.0:1789760798.167888:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777184788800 time 777235664058 state 0x0 ref 1 00000100:00100000:1.0:1789760798.167902:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777235924474 time 777235664058 state 0x1 ref 1 00000100:00100000:1.0:1789760798.167936:0:14638:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777235924474 time 777272646922 state 0x0 ref 1 00000100:00100000:1.0:1789760798.167954:0:14638:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508497920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760798.168009:0:14638:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25907200, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:1.0:1789760798.168018:0:14638:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57402 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.168031:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760798.168035:0:14638:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.168041:0:14638:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57402 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.168573:0:14638:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57402 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.193127:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973645 is committed 00002000:00010000:3.0:1789760798.193279:0:14638:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25907200 -> 25911296 00010000:00010000:3.0:1789760798.193312:0:14638:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57402 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.193324:0:14638:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57402 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.193333:0:14638:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57402 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.193340:0:14638:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.193345:0:14638:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57402 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.193354:0:14638:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57402 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.193374:0:14638:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57402 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.193382:0:14638:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe57402 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25907200->25911295] (req 25907200->25911295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14638 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.193469:0:14638:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508497920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25517us (25765us total) trans 4294973645 rc 0/0 00000100:00100000:3.0:1789760798.193480:0:14638:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6357 00000100:00100000:3.0:1789760798.193490:0:14638:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777235924474 time 777272646922 state 0x2 ref 0 00000100:00100000:3.0:1789760798.195647:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760798.195737:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508498048 00000100:00100000:1.0:1789760798.195747:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508498048 00000100:00100000:1.0:1789760798.195795:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777235924474 time 777272646922 state 0x0 ref 1 00000100:00100000:1.0:1789760798.195809:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777272907338 time 777272646922 state 0x1 ref 1 00000100:00100000:1.0:1789760798.195843:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777272907338 time 777300552958 state 0x0 ref 1 00000100:00100000:1.0:1789760798.195857:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508498048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760798.195962:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25911296, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:1.0:1789760798.195971:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57409 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.195992:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760798.195996:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.196062:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57409 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.196077:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57409 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.222590:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973646 is committed 00002000:00010000:3.0:1789760798.222689:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25911296 -> 25915392 00010000:00010000:3.0:1789760798.222704:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57409 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.222721:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57409 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.222730:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57409 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.222738:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.222743:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57409 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.222751:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57409 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.222770:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57409 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.222778:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe57409 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25911296->25915391] (req 25911296->25915391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.222857:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508498048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26997us (27213us total) trans 4294973646 rc 0/0 00000100:00100000:3.0:1789760798.222866:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6358 00000100:00100000:3.0:1789760798.222876:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777272907338 time 777300552958 state 0x2 ref 0 00000100:00100000:3.0:1789760798.226191:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760798.226325:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508498176 00000100:00100000:0.0:1789760798.226335:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508498176 00000100:00100000:0.0:1789760798.226394:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777272907338 time 777300552958 state 0x0 ref 1 00000100:00100000:0.0:1789760798.226409:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777300813374 time 777300552958 state 0x1 ref 1 00000100:00100000:0.0:1789760798.226441:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777300813374 time 777331151698 state 0x0 ref 1 00000100:00100000:0.0:1789760798.226454:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508498176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760798.226507:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25915392, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:0.0:1789760798.226516:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57410 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.226530:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760798.226535:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.226541:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57410 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.226554:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57410 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.248525:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973647 is committed 00002000:00010000:3.0:1789760798.248674:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25915392 -> 25919488 00010000:00010000:3.0:1789760798.248689:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57410 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.248701:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57410 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.248710:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57410 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.248717:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.248722:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57410 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.248731:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57410 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.248752:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57410 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.248760:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57410 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25915392->25919487] (req 25915392->25919487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.248847:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508498176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22390us (22660us total) trans 4294973647 rc 0/0 00000100:00100000:3.0:1789760798.248857:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6359 00000100:00100000:3.0:1789760798.248866:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777300813374 time 777331151698 state 0x2 ref 0 00000100:00100000:2.0:1789760798.260362:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.260482:0:14638:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508498304 00000100:00100000:3.0:1789760798.260491:0:14638:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508498304 00000100:00100000:3.0:1789760798.260543:0:14638:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777300813374 time 777331151698 state 0x0 ref 1 00000100:00100000:3.0:1789760798.260558:0:14638:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777331412114 time 777331151698 state 0x1 ref 1 00000100:00100000:2.0:1789760798.260668:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777331412114 time 777365375895 state 0x0 ref 1 00000100:00100000:2.0:1789760798.260682:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508498304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760798.260729:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25919488, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:2.0:1789760798.260738:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe57417 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.260752:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760798.260756:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.260762:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe57417 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.260779:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe57417 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.290417:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973648 is committed 00002000:00010000:2.0:1789760798.290543:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25919488 -> 25923584 00010000:00010000:2.0:1789760798.290558:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe57417 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.290580:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe57417 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.290589:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe57417 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.290596:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760798.290601:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe57417 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.290609:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe57417 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.290627:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe57417 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.290635:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9d2ae2/0xb49177c0efe57417 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25919488->25923583] (req 25919488->25923583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.290740:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508498304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 30057us (30383us total) trans 4294973648 rc 0/0 00000100:00100000:2.0:1789760798.290750:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6360 00000100:00100000:2.0:1789760798.290760:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777331412114 time 777365375895 state 0x2 ref 0 00000100:00100000:0.0:1789760798.295500:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.295612:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508498432 00000100:00100000:3.0:1789760798.295622:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508498432 00000100:00100000:3.0:1789760798.295696:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777331412114 time 777365375895 state 0x0 ref 1 00000100:00100000:3.0:1789760798.295710:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777365636311 time 777365375895 state 0x1 ref 1 00000100:00100000:3.0:1789760798.295739:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777365636311 time 777400449404 state 0x0 ref 1 00000100:00100000:3.0:1789760798.295750:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e60380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508498432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.295794:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25923584, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:3.0:1789760798.295802:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5741e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.295816:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.295820:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.295826:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5741e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.295839:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5741e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760798.331417:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25923584 -> 25927680 00010000:00010000:3.0:1789760798.331435:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5741e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.331448:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5741e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.331456:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5741e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.331463:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.331468:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5741e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.331477:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5741e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.331496:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5741e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.331503:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5741e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25923584->25927679] (req 25923584->25927679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.331580:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e60380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508498432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35826us (36083us total) trans 4294973649 rc 0/0 00000100:00100000:3.0:1789760798.331603:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6361 00000100:00100000:3.0:1789760798.331612:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777365636311 time 777400449404 state 0x2 ref 0 00000001:00080000:3.0:1789760798.331666:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973649 is committed 00000100:00100000:3.0:1789760798.346112:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760798.346281:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508498560 00000100:00100000:0.0:1789760798.346441:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508498560 00000100:00100000:2.0:1789760798.346565:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777365636311 time 777400449404 state 0x0 ref 1 00000100:00100000:2.0:1789760798.346582:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777400709820 time 777400449404 state 0x1 ref 1 00000100:00100000:2.0:1789760798.346657:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777400709820 time 777451357376 state 0x0 ref 1 00000100:00100000:2.0:1789760798.346671:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508498560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760798.346734:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25927680, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:2.0:1789760798.346743:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57425 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.346758:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760798.346763:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.346770:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57425 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.346785:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57425 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.381357:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973650 is committed 00002000:00010000:2.0:1789760798.381476:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25927680 -> 25931776 00010000:00010000:2.0:1789760798.381491:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57425 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.381503:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57425 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.381511:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57425 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.381517:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760798.381522:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57425 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.381531:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57425 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.381549:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57425 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.381556:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005101d87e/0xb49177c0efe57425 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25927680->25931775] (req 25927680->25931775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.381797:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508498560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35123us (35689us total) trans 4294973650 rc 0/0 00000100:00100000:2.0:1789760798.381808:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6362 00000100:00100000:2.0:1789760798.381817:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777400709820 time 777451357376 state 0x2 ref 0 00000100:00100000:2.0:1789760798.388862:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760798.389077:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508498688 00000100:00100000:0.0:1789760798.389087:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508498688 00000100:00100000:0.0:1789760798.389141:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777400709820 time 777451357376 state 0x0 ref 1 00000100:00100000:0.0:1789760798.389157:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777451617792 time 777451357376 state 0x1 ref 1 00000100:00100000:0.0:1789760798.389187:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777451617792 time 777493897889 state 0x0 ref 1 00000100:00100000:0.0:1789760798.389199:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508498688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760798.389243:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25931776, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:0.0:1789760798.389252:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5742c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.389267:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:0.0:1789760798.389271:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.389276:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5742c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.389368:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5742c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760798.416417:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25931776 -> 25935872 00010000:00010000:1.0:1789760798.416438:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5742c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.416451:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5742c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.416460:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5742c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.416467:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760798.416471:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5742c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.416482:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5742c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.416502:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5742c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.416511:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5742c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25931776->25935871] (req 25931776->25935871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.416696:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508498688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27493us (27838us total) trans 4294973651 rc 0/0 00000100:00100000:1.0:1789760798.416707:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6363 00000100:00100000:1.0:1789760798.416717:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777451617792 time 777493897889 state 0x2 ref 0 00000001:00080000:3.0:1789760798.416793:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973651 is committed 00000100:00100000:0.0:1789760798.423766:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.423850:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508498816 00000100:00100000:3.0:1789760798.423860:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508498816 00000100:00100000:3.0:1789760798.423911:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777451617792 time 777493897889 state 0x0 ref 1 00000100:00100000:3.0:1789760798.423927:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777494158305 time 777493897889 state 0x1 ref 1 00000100:00100000:3.0:1789760798.423991:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777494158305 time 777528700893 state 0x0 ref 1 00000100:00100000:3.0:1789760798.424011:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e62d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508498816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.424056:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25935872, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:3.0:1789760798.424065:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57433 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.424080:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.424094:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.424101:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57433 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.424117:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57433 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.448555:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973652 is committed 00002000:00010000:3.0:1789760798.448656:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25935872 -> 25939968 00010000:00010000:3.0:1789760798.448671:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57433 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.448683:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57433 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.448693:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57433 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.448700:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.448705:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57433 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.448714:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57433 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.448734:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57433 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.448742:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57433 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25935872->25939967] (req 25935872->25939967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.448831:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e62d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508498816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24824us (25069us total) trans 4294973652 rc 0/0 00000100:00100000:3.0:1789760798.448846:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6364 00000100:00100000:3.0:1789760798.448855:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777494158305 time 777528700893 state 0x2 ref 0 00000100:00100000:0.0:1789760798.452119:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760798.452238:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508498944 00000100:00100000:2.0:1789760798.452248:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508498944 00000100:00100000:2.0:1789760798.452394:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777494158305 time 777528700893 state 0x0 ref 1 00000100:00100000:2.0:1789760798.452408:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777528961309 time 777528700893 state 0x1 ref 1 00000100:00100000:2.0:1789760798.452436:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777528961309 time 777557146092 state 0x0 ref 1 00000100:00100000:2.0:1789760798.452450:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508498944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760798.452493:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25939968, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:2.0:1789760798.452502:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5743a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.452518:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:2.0:1789760798.452523:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.452529:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5743a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.452543:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5743a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.462469:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760798.476759:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973653 is committed 00002000:00010000:3.0:1789760798.476977:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25939968 -> 25944064 00010000:00010000:3.0:1789760798.476993:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5743a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.477006:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5743a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.477015:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5743a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.477021:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.477025:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5743a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.477034:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5743a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.477051:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5743a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.477059:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e234f6cb/0xb49177c0efe5743a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25939968->25944063] (req 25939968->25944063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.477220:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e62300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508498944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24769us (25110us total) trans 4294973653 rc 0/0 00000100:00100000:3.0:1789760798.477229:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6365 00000100:00100000:3.0:1789760798.477238:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777528961309 time 777557146092 state 0x2 ref 0 00000100:00100000:2.0:1789760798.483436:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760798.483835:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508499072 00000100:00100000:1.0:1789760798.483844:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508499072 00000100:00100000:1.0:1789760798.483891:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777528961309 time 777557146092 state 0x0 ref 1 00000100:00100000:1.0:1789760798.483905:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777557406508 time 777557146092 state 0x1 ref 1 00000100:00100000:1.0:1789760798.483931:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777557406508 time 777588642555 state 0x0 ref 1 00000100:00100000:1.0:1789760798.483942:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508499072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760798.483984:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25944064, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0xc807 00010000:00010000:1.0:1789760798.483993:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe57441 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.484014:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760798.484018:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.484023:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe57441 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.484037:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe57441 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.509624:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973654 is committed 00002000:00010000:3.0:1789760798.509816:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25944064 -> 25948160 00002000:00010000:3.0:1789760798.509822:0:8108:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760798 -> 1789760799 00002000:00010000:3.0:1789760798.509825:0:8108:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760798 -> 1789760799 00010000:00010000:3.0:1789760798.509838:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe57441 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.509850:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe57441 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.509859:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe57441 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.509867:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.509872:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe57441 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.509881:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe57441 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.509906:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe57441 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.509913:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe57441 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25944064->25948159] (req 25944064->25948159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.510010:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508499072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 26056us (26571us total) trans 4294973654 rc 0/0 00000100:00100000:3.0:1789760798.510020:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6366 00000100:00100000:3.0:1789760798.510029:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777557406508 time 777588642555 state 0x2 ref 0 00000100:00100000:0.0:1789760798.516999:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.517092:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508499200 00000100:00100000:3.0:1789760798.517102:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508499200 00000100:00100000:3.0:1789760798.517156:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777557406508 time 777588642555 state 0x0 ref 1 00000100:00100000:3.0:1789760798.517171:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777588902971 time 777588642555 state 0x1 ref 1 00000100:00100000:3.0:1789760798.517204:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777588902971 time 777621914739 state 0x0 ref 1 00000100:00100000:3.0:1789760798.517217:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e63100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508499200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.517266:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25948160, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760798.517275:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57448 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.517325:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.517330:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.517337:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57448 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.517357:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57448 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.543895:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973655 is committed 00002000:00010000:3.0:1789760798.544190:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25948160 -> 25952256 00010000:00010000:3.0:1789760798.544205:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57448 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.544217:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57448 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.544226:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57448 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.544234:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.544239:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57448 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.544248:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57448 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.544266:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57448 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.544273:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57448 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25948160->25952255] (req 25948160->25952255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.544425:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e63100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508499200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27205us (27431us total) trans 4294973655 rc 0/0 00000100:00100000:3.0:1789760798.544434:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6367 00000100:00100000:3.0:1789760798.544443:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777588902971 time 777621914739 state 0x2 ref 0 00000100:00100000:0.0:1789760798.548025:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.548087:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508499328 00000100:00100000:3.0:1789760798.548097:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508499328 00000100:00100000:3.0:1789760798.548152:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777588902971 time 777621914739 state 0x0 ref 1 00000100:00100000:3.0:1789760798.548167:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777622175155 time 777621914739 state 0x1 ref 1 00000100:00100000:3.0:1789760798.548193:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777622175155 time 777652903737 state 0x0 ref 1 00000100:00100000:3.0:1789760798.548207:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e61180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508499328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.548252:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25952256, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760798.548260:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5744f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.548275:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.548280:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.548311:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5744f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.548327:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5744f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.567790:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973656 is committed 00002000:00010000:3.0:1789760798.567891:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25952256 -> 25956352 00010000:00010000:3.0:1789760798.567907:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5744f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.567919:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5744f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.567928:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5744f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.567935:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.567940:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5744f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.567949:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5744f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.567966:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5744f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.567972:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5744f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25952256->25956351] (req 25952256->25956351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.568095:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e61180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508499328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19887us (20074us total) trans 4294973656 rc 0/0 00000100:00100000:3.0:1789760798.568105:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6368 00000100:00100000:3.0:1789760798.568112:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777622175155 time 777652903737 state 0x2 ref 0 00000100:00100000:2.0:1789760798.572187:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.572322:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508499456 00000100:00100000:3.0:1789760798.572333:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508499456 00000100:00100000:3.0:1789760798.572380:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777622175155 time 777652903737 state 0x0 ref 1 00000100:00100000:3.0:1789760798.572394:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777653164153 time 777652903737 state 0x1 ref 1 00000100:00100000:3.0:1789760798.572422:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777653164153 time 777677133058 state 0x0 ref 1 00000100:00100000:3.0:1789760798.572435:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508499456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.572485:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25956352, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760798.572495:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57456 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.572510:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.572515:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.572522:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57456 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.572535:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57456 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.590550:0:3475:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789760798.590571:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa048846d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534095744:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760798.590635:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760798.590788:0:7358:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534095744 00000100:00100000:1.0:1789760798.590796:0:7358:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534095744 00000100:00100000:1.0:1789760798.590841:0:7358:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 476 00000100:00100000:1.0:1789760798.590850:0:7358:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e63b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534095744:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760798.590886:0:7358:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789760798.590939:0:7358:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e63b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3475:x1876699534095744:12345-0@lo:41:osp-pre-0-1.0 Request processed in 89us (302us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760798.590945:0:7358:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 476 00000100:00100000:0.0:1789760798.591029:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa048846d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3475:1876699534095744:0@lo:41:osp-pre-0-1.0 00002000:00010000:3.0:1789760798.596338:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25956352 -> 25960448 00010000:00010000:3.0:1789760798.596362:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57456 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.596373:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57456 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.596376:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973657 is committed 00010000:00010000:3.0:1789760798.596387:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57456 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.596395:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.596400:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57456 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.596414:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57456 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.596436:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57456 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.596442:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57456 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25956352->25960447] (req 25956352->25960447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.596529:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508499456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24091us (24347us total) trans 4294973657 rc 0/0 00000100:00100000:3.0:1789760798.596539:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6369 00000100:00100000:3.0:1789760798.596548:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777653164153 time 777677133058 state 0x2 ref 0 00000100:00100000:0.0:1789760798.599699:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.599836:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508499584 00000100:00100000:3.0:1789760798.599845:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508499584 00000100:00100000:3.0:1789760798.599911:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777653164153 time 777677133058 state 0x0 ref 1 00000100:00100000:3.0:1789760798.599927:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777677393474 time 777677133058 state 0x1 ref 1 00000100:00100000:3.0:1789760798.600072:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777677393474 time 777704781113 state 0x0 ref 1 00000100:00100000:3.0:1789760798.600086:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508499584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.600135:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25960448, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760798.600144:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5745d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.600159:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.600163:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.600169:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5745d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.600184:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5745d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.622072:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973658 is committed 00002000:00010000:3.0:1789760798.622109:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25960448 -> 25964544 00002000:00010000:3.0:1789760798.622115:0:8110:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 51207 -> 51209 00010000:00010000:3.0:1789760798.622127:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5745d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.622139:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5745d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.622152:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5745d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.622159:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.622164:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5745d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.622172:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5745d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.622191:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5745d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.622199:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5745d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25960448->25964543] (req 25960448->25964543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.622367:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508499584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22198us (22591us total) trans 4294973658 rc 0/0 00000100:00100000:3.0:1789760798.622378:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6370 00000100:00100000:3.0:1789760798.622388:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777677393474 time 777704781113 state 0x2 ref 0 00000100:00100000:0.0:1789760798.626840:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760798.626935:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508499712 00000100:00100000:1.0:1789760798.626946:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508499712 00000100:00100000:1.0:1789760798.626998:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777677393474 time 777704781113 state 0x0 ref 1 00000100:00100000:1.0:1789760798.627018:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777705041529 time 777704781113 state 0x1 ref 1 00000100:00100000:1.0:1789760798.627091:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777705041529 time 777731800485 state 0x0 ref 1 00000100:00100000:1.0:1789760798.627104:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508499712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760798.627151:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25964544, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc809 00010000:00010000:1.0:1789760798.627161:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57464 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.627177:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760798.627182:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.627188:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57464 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.627202:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57464 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.647836:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973659 is committed 00002000:00010000:1.0:1789760798.647909:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25964544 -> 25968640 00010000:00010000:1.0:1789760798.647921:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57464 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.647930:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57464 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.647935:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57464 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.647939:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760798.647943:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57464 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.647949:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57464 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.647962:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57464 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.647968:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57464 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25964544->25968639] (req 25964544->25968639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.648059:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508499712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20953us (21224us total) trans 4294973659 rc 0/0 00000100:00100000:1.0:1789760798.648068:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6371 00000100:00100000:1.0:1789760798.648077:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777705041529 time 777731800485 state 0x2 ref 0 00000100:00100000:2.0:1789760798.651932:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.652020:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508499840 00000100:00100000:3.0:1789760798.652040:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508499840 00000100:00100000:3.0:1789760798.652103:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777705041529 time 777731800485 state 0x0 ref 1 00000100:00100000:3.0:1789760798.652120:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777732060901 time 777731800485 state 0x1 ref 1 00000100:00100000:3.0:1789760798.652146:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777732060901 time 777756856469 state 0x0 ref 1 00000100:00100000:3.0:1789760798.652159:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508499840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.652245:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25968640, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc809 00010000:00010000:3.0:1789760798.652325:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5746b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.652340:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.652344:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.652350:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5746b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.652365:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5746b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760798.679375:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25968640 -> 25972736 00010000:00010000:3.0:1789760798.679395:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5746b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.679409:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5746b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.679417:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5746b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.679431:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.679436:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5746b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.679445:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5746b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.679465:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5746b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.679473:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5746b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25968640->25972735] (req 25968640->25972735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.679564:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508499840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27402us (27635us total) trans 4294973660 rc 0/0 00000100:00100000:3.0:1789760798.679574:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6372 00000100:00100000:3.0:1789760798.679593:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777732060901 time 777756856469 state 0x2 ref 0 00000001:00080000:3.0:1789760798.679629:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973660 is committed 00000100:00100000:0.0:1789760798.683540:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.683636:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508499968 00000100:00100000:3.0:1789760798.683660:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508499968 00000100:00100000:3.0:1789760798.683716:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777732060901 time 777756856469 state 0x0 ref 1 00000100:00100000:3.0:1789760798.683730:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777757116885 time 777756856469 state 0x1 ref 1 00000100:00100000:3.0:1789760798.683757:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777757116885 time 777788468478 state 0x0 ref 1 00000100:00100000:3.0:1789760798.683782:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508499968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.683831:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25972736, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc809 00010000:00010000:3.0:1789760798.683840:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57472 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.683860:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.683864:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.683871:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57472 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.683885:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57472 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760798.720771:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25972736 -> 25976832 00010000:00010000:3.0:1789760798.720795:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57472 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.720808:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57472 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.720817:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57472 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.720824:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.720829:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57472 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.720837:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57472 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.720857:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57472 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.720866:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57472 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25972736->25976831] (req 25972736->25976831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.721670:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e63800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508499968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37192us (37429us total) trans 4294973661 rc 0/0 00000100:00100000:3.0:1789760798.721683:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6373 00000100:00100000:3.0:1789760798.721701:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777757116885 time 777788468478 state 0x2 ref 0 00000001:00080000:3.0:1789760798.721741:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973661 is committed 00000100:00100000:0.0:1789760798.723446:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760798.723613:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508500096 00000100:00100000:1.0:1789760798.723623:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508500096 00000100:00100000:1.0:1789760798.723684:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777757116885 time 777788468478 state 0x0 ref 1 00000100:00100000:1.0:1789760798.723699:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777788728894 time 777788468478 state 0x1 ref 1 00000100:00100000:1.0:1789760798.723729:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777788728894 time 777828439283 state 0x0 ref 1 00000100:00100000:1.0:1789760798.723742:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508500096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760798.723786:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25976832, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:1.0:1789760798.723795:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57479 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.723810:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:1.0:1789760798.723814:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.723820:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57479 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.723838:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57479 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760798.755473:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25976832 -> 25980928 00010000:00010000:3.0:1789760798.755495:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57479 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.755508:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57479 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.755517:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57479 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.755530:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.755535:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57479 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.755543:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57479 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.755563:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57479 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.755570:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57479 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25976832->25980927] (req 25976832->25980927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.755681:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508500096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 31935us (32236us total) trans 4294973662 rc 0/0 00000100:00100000:3.0:1789760798.755692:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6374 00000100:00100000:3.0:1789760798.755703:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777788728894 time 777828439283 state 0x2 ref 0 00000001:00080000:3.0:1789760798.755743:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973662 is committed 00000100:00100000:2.0:1789760798.758390:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.758428:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508500224 00000100:00100000:3.0:1789760798.758438:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508500224 00000100:00100000:3.0:1789760798.758491:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777788728894 time 777828439283 state 0x0 ref 1 00000100:00100000:3.0:1789760798.758507:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777828699699 time 777828439283 state 0x1 ref 1 00000100:00100000:3.0:1789760798.758540:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777828699699 time 777863250764 state 0x0 ref 1 00000100:00100000:3.0:1789760798.758554:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508500224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.758611:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25980928, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760798.758620:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57480 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.758636:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.758640:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.758658:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57480 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.758672:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57480 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760798.806415:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25980928 -> 25985024 00010000:00010000:3.0:1789760798.806435:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57480 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.806449:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57480 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.806458:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57480 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.806466:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.806471:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57480 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.806480:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57480 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.806499:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57480 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.806507:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57480 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25980928->25985023] (req 25980928->25985023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.806890:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b66d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508500224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 48331us (48502us total) trans 4294973663 rc 0/0 00000100:00100000:3.0:1789760798.806901:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6375 00000100:00100000:3.0:1789760798.806910:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777828699699 time 777863250764 state 0x2 ref 0 00000001:00080000:3.0:1789760798.806949:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973663 is committed 00000100:00100000:0.0:1789760798.812056:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.812702:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508500352 00000100:00100000:3.0:1789760798.812713:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508500352 00000100:00100000:3.0:1789760798.812765:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777828699699 time 777863250764 state 0x0 ref 1 00000100:00100000:3.0:1789760798.812782:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777863511180 time 777863250764 state 0x1 ref 1 00000100:00100000:3.0:1789760798.812812:0:8110:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777863511180 time 777917522212 state 0x0 ref 1 00000100:00100000:3.0:1789760798.812824:0:8110:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508500352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.812867:0:8110:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25985024, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760798.812876:0:8110:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57487 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.812892:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.812897:0:8110:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.812902:0:8110:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57487 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.812915:0:8110:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57487 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760798.842446:0:8110:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25985024 -> 25989120 00010000:00010000:3.0:1789760798.842465:0:8110:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57487 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.842478:0:8110:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57487 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.842487:0:8110:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57487 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.842494:0:8110:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.842498:0:8110:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57487 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.842507:0:8110:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57487 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.842532:0:8110:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57487 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.842539:0:8110:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57487 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25985024->25989119] (req 25985024->25989119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8110 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.842638:0:8110:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e61500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2406:x1876699508500352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29810us (30586us total) trans 4294973664 rc 0/0 00000100:00100000:3.0:1789760798.842656:0:8110:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6376 00000100:00100000:3.0:1789760798.842666:0:8110:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777863511180 time 777917522212 state 0x2 ref 0 00000001:00080000:2.0:1789760798.842881:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973664 is committed 00000100:00100000:0.0:1789760798.850406:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.850691:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508500480 00000100:00100000:3.0:1789760798.850701:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508500480 00000100:00100000:3.0:1789760798.850756:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777863511180 time 777917522212 state 0x0 ref 1 00000100:00100000:3.0:1789760798.850772:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777917782628 time 777917522212 state 0x1 ref 1 00000100:00100000:3.0:1789760798.850802:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777917782628 time 777955512681 state 0x0 ref 1 00000100:00100000:3.0:1789760798.850816:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e61c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508500480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.850866:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25989120, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760798.850875:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5748e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.850890:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.850895:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.850901:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5748e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.850916:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5748e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.871396:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973665 is committed 00002000:00010000:3.0:1789760798.871469:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25989120 -> 25993216 00010000:00010000:3.0:1789760798.871484:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5748e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.871496:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5748e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.871504:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5748e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.871511:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.871516:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5748e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.871524:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5748e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.871544:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5748e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.871567:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe5748e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25989120->25993215] (req 25989120->25993215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.871663:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e61c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508500480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20845us (21262us total) trans 4294973665 rc 0/0 00000100:00100000:3.0:1789760798.871673:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6377 00000100:00100000:3.0:1789760798.871680:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777917782628 time 777955512681 state 0x2 ref 0 00000100:00100000:1.0:1789760798.874408:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.874446:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508500608 00000100:00100000:3.0:1789760798.874455:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508500608 00000100:00100000:3.0:1789760798.874497:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777917782628 time 777955512681 state 0x0 ref 1 00000100:00100000:3.0:1789760798.874508:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777955773097 time 777955512681 state 0x1 ref 1 00000100:00100000:3.0:1789760798.874530:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777955773097 time 777979241474 state 0x0 ref 1 00000100:00100000:3.0:1789760798.874735:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508500608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.874775:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25993216, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760798.874781:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57495 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.874793:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=777s v=(0 0) 00010000:00010000:3.0:1789760798.874796:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.874800:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57495 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.874814:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57495 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.895716:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973666 is committed 00002000:00010000:3.0:1789760798.895798:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25993216 -> 25997312 00010000:00010000:3.0:1789760798.895818:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57495 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.895830:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57495 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.895839:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57495 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.895846:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.895850:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57495 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.895859:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57495 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.896141:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57495 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.896151:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57495 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25993216->25997311] (req 25993216->25997311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.896247:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508500608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 21565us (21843us total) trans 4294973666 rc 0/0 00000100:00100000:3.0:1789760798.897161:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6378 00000100:00100000:3.0:1789760798.897171:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777955773097 time 777979241474 state 0x2 ref 0 00000100:00100000:1.0:1789760798.902125:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.902220:0:8110:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508500736 00000100:00100000:3.0:1789760798.902230:0:8110:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508500736 00000100:00100000:3.0:1789760798.902387:0:8110:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777955773097 time 777979241474 state 0x0 ref 1 00000100:00100000:3.0:1789760798.902406:0:8110:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777979501890 time 777979241474 state 0x1 ref 1 00000100:00100000:0.0:1789760798.902516:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777979501890 time 778007223804 state 0x0 ref 1 00000100:00100000:0.0:1789760798.902530:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508500736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760798.902575:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 25997312, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:0.0:1789760798.902583:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5749c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.902599:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760798.902603:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.902610:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5749c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.902625:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5749c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.924138:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973667 is committed 00002000:00010000:1.0:1789760798.924375:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 25997312 -> 26001408 00010000:00010000:1.0:1789760798.924390:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5749c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.924402:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5749c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.924411:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5749c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.924418:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760798.924422:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5749c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.924431:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5749c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.924450:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5749c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.924463:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe5749c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [25997312->26001407] (req 25997312->26001407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.924585:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508500736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22054us (22709us total) trans 4294973667 rc 0/0 00000100:00100000:1.0:1789760798.924595:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6379 00000100:00100000:1.0:1789760798.924604:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777979501890 time 778007223804 state 0x2 ref 0 00000100:00100000:0.0:1789760798.928491:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.928609:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508500864 00000100:00100000:3.0:1789760798.928620:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508500864 00000100:00100000:3.0:1789760798.928698:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 777979501890 time 778007223804 state 0x0 ref 1 00000100:00100000:3.0:1789760798.928713:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778007484220 time 778007223804 state 0x1 ref 1 00000100:00100000:3.0:1789760798.928745:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778007484220 time 778033455714 state 0x0 ref 1 00000100:00100000:3.0:1789760798.928768:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508500864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.928821:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26001408, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760798.928830:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe574a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.928844:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760798.928849:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.928856:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe574a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.928870:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe574a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760798.948730:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26001408 -> 26005504 00000001:00080000:3.0:1789760798.948748:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973668 is committed 00010000:00010000:2.0:1789760798.948806:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe574a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.948819:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe574a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.948827:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe574a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.948835:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760798.948840:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe574a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.948849:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe574a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.948868:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe574a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.948876:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe574a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26001408->26005503] (req 26001408->26005503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.948999:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508500864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20228us (20513us total) trans 4294973668 rc 0/0 00000100:00100000:2.0:1789760798.949011:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6380 00000100:00100000:2.0:1789760798.949022:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778007484220 time 778033455714 state 0x2 ref 0 00000100:00100000:2.0:1789760798.954418:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760798.954516:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508500992 00000100:00100000:3.0:1789760798.954525:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508500992 00000100:00100000:3.0:1789760798.954576:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778007484220 time 778033455714 state 0x0 ref 1 00000100:00100000:3.0:1789760798.954592:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778033716130 time 778033455714 state 0x1 ref 1 00000100:00100000:3.0:1789760798.954660:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778033716130 time 778059370179 state 0x0 ref 1 00000100:00100000:3.0:1789760798.954673:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508500992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760798.954718:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26005504, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760798.954727:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.954741:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760798.954746:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.954753:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.954767:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.973765:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973669 is committed 00002000:00010000:3.0:1789760798.973953:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26005504 -> 26009600 00010000:00010000:3.0:1789760798.973966:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.973975:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.973982:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.973986:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760798.973989:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.973997:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.974016:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.974024:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26005504->26009599] (req 26005504->26009599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.974101:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508500992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19426us (19688us total) trans 4294973669 rc 0/0 00000100:00100000:3.0:1789760798.974115:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6381 00000100:00100000:3.0:1789760798.974123:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778033716130 time 778059370179 state 0x2 ref 0 00000100:00100000:0.0:1789760798.979242:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760798.980800:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508501120 00000100:00100000:1.0:1789760798.980813:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508501120 00000100:00100000:1.0:1789760798.980869:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778033716130 time 778059370179 state 0x0 ref 1 00000100:00100000:1.0:1789760798.980884:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778059630595 time 778059370179 state 0x1 ref 1 00000100:00100000:1.0:1789760798.980918:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778059630595 time 778085628135 state 0x0 ref 1 00000100:00100000:1.0:1789760798.980930:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e60000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508501120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760798.980978:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26009600, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:1.0:1789760798.980987:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe574b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.981255:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760798.981259:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.981266:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe574b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.981347:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe574b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760799.003017:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973670 is committed 00002000:00010000:0.0:1789760799.003075:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26009600 -> 26013696 00010000:00010000:0.0:1789760799.003090:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe574b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.003104:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe574b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.003112:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe574b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.003119:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760799.003124:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe574b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.003132:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe574b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.003151:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe574b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.003159:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe574b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26009600->26013695] (req 26009600->26013695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760799.003417:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e60000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508501120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22484us (24181us total) trans 4294973670 rc 0/0 00000100:00100000:0.0:1789760799.003427:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6382 00000100:00100000:0.0:1789760799.003437:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778059630595 time 778085628135 state 0x2 ref 0 00000100:00100000:0.0:1789760799.006514:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760799.006613:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508501248 00000100:00100000:2.0:1789760799.006622:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508501248 00000100:00100000:2.0:1789760799.006780:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778059630595 time 778085628135 state 0x0 ref 1 00000100:00100000:2.0:1789760799.006795:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778085888551 time 778085628135 state 0x1 ref 1 00000100:00100000:2.0:1789760799.006827:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778085888551 time 778111537508 state 0x0 ref 1 00000100:00100000:2.0:1789760799.006840:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508501248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760799.006885:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26013696, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:2.0:1789760799.006895:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe574b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.006910:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760799.006914:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760799.006920:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe574b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.006933:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe574b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760799.007680:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760799.047131:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973671 is committed 00002000:00010000:0.0:1789760799.047193:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26013696 -> 26017792 00010000:00010000:0.0:1789760799.047211:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe574b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.047224:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe574b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.047233:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe574b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.047240:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760799.047245:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe574b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.047254:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe574b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.047274:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe574b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.047282:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe574b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26013696->26017791] (req 26013696->26017791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.047439:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508501248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 40596us (40929us total) trans 4294973671 rc 0/0 00000100:00100000:1.0:1789760799.047450:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6383 00000100:00100000:1.0:1789760799.047459:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778085888551 time 778111537508 state 0x2 ref 0 00000100:00100000:2.0:1789760799.050249:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760799.050871:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508501376 00000100:00100000:3.0:1789760799.050881:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508501376 00000100:00100000:3.0:1789760799.050933:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778085888551 time 778111537508 state 0x0 ref 1 00000100:00100000:3.0:1789760799.050948:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778111797924 time 778111537508 state 0x1 ref 1 00000100:00100000:3.0:1789760799.050982:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778111797924 time 778155692165 state 0x0 ref 1 00000100:00100000:3.0:1789760799.050994:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508501376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760799.051057:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26017792, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760799.051067:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.051084:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760799.051087:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760799.051091:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.051103:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.073924:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973672 is committed 00002000:00010000:3.0:1789760799.073967:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26017792 -> 26021888 00010000:00010000:3.0:1789760799.073983:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.073997:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.074174:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.074184:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760799.074188:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.074207:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.074227:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.074235:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26017792->26021887] (req 26017792->26021887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760799.074375:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508501376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23378us (24128us total) trans 4294973672 rc 0/0 00000100:00100000:3.0:1789760799.074386:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6384 00000100:00100000:3.0:1789760799.074395:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778111797924 time 778155692165 state 0x2 ref 0 00000100:00100000:0.0:1789760799.081506:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.081673:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508501504 00000100:00100000:1.0:1789760799.081684:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508501504 00000100:00100000:1.0:1789760799.081741:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778111797924 time 778155692165 state 0x0 ref 1 00000100:00100000:1.0:1789760799.081757:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778155952581 time 778155692165 state 0x1 ref 1 00000100:00100000:1.0:1789760799.081784:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778155952581 time 778186494362 state 0x0 ref 1 00000100:00100000:1.0:1789760799.081798:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508501504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.081845:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26021888, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:1.0:1789760799.081855:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.081871:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.081876:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.081882:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.081903:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760799.113980:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973673 is committed 00002000:00010000:1.0:1789760799.114134:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26021888 -> 26025984 00010000:00010000:1.0:1789760799.114150:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.114163:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.114171:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.114178:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.114183:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.114191:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.114210:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.114218:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26021888->26025983] (req 26021888->26025983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.114497:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508501504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 32697us (32992us total) trans 4294973673 rc 0/0 00000100:00100000:1.0:1789760799.114506:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6385 00000100:00100000:1.0:1789760799.114516:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778155952581 time 778186494362 state 0x2 ref 0 00000100:00100000:0.0:1789760799.121735:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.121783:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508501632 00000100:00100000:1.0:1789760799.121793:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508501632 00000100:00100000:1.0:1789760799.121846:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778155952581 time 778186494362 state 0x0 ref 1 00000100:00100000:1.0:1789760799.121861:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778186754778 time 778186494362 state 0x1 ref 1 00000100:00100000:1.0:1789760799.121886:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778186754778 time 778226596784 state 0x0 ref 1 00000100:00100000:1.0:1789760799.121896:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508501632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.121984:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26025984, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:1.0:1789760799.121991:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.122004:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.122007:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.122012:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.122021:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760799.146560:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973674 is committed 00002000:00010000:1.0:1789760799.146592:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26025984 -> 26030080 00010000:00010000:1.0:1789760799.146608:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.146617:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.146623:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.146628:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.146632:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.146650:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.146671:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.146679:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26025984->26030079] (req 26025984->26030079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.146777:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508501632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24878us (25048us total) trans 4294973674 rc 0/0 00000100:00100000:1.0:1789760799.146791:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6386 00000100:00100000:1.0:1789760799.146801:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778186754778 time 778226596784 state 0x2 ref 0 00000100:00100000:2.0:1789760799.154525:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760799.156211:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508501760 00000100:00100000:3.0:1789760799.156225:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508501760 00000100:00100000:3.0:1789760799.156300:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778186754778 time 778226596784 state 0x0 ref 1 00000100:00100000:3.0:1789760799.156318:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778226857200 time 778226596784 state 0x1 ref 1 00000100:00100000:3.0:1789760799.156352:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778226857200 time 778261063283 state 0x0 ref 1 00000100:00100000:3.0:1789760799.156366:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508501760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760799.156410:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26030080, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760799.156421:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.156436:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760799.156441:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760799.156447:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.156461:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760799.174283:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973675 is committed 00002000:00010000:1.0:1789760799.174524:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26030080 -> 26034176 00010000:00010000:1.0:1789760799.174539:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.174553:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.174562:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.174569:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.174573:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.174592:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.174610:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.174617:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe574d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26030080->26034175] (req 26030080->26034175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.174728:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508501760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 18361us (20208us total) trans 4294973675 rc 0/0 00000100:00100000:1.0:1789760799.174738:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6387 00000100:00100000:1.0:1789760799.174747:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778226857200 time 778261063283 state 0x2 ref 0 00000100:00100000:0.0:1789760799.179928:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.180226:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508501888 00000100:00100000:1.0:1789760799.180235:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508501888 00000100:00100000:1.0:1789760799.180319:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778226857200 time 778261063283 state 0x0 ref 1 00000100:00100000:1.0:1789760799.180334:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778261323699 time 778261063283 state 0x1 ref 1 00000100:00100000:1.0:1789760799.180356:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778261323699 time 778285066645 state 0x0 ref 1 00000100:00100000:1.0:1789760799.180369:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508501888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.180415:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26034176, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:1.0:1789760799.180423:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.180437:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.180442:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.180448:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.180461:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760799.194999:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973676 is committed 00002000:00010000:1.0:1789760799.195094:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26034176 -> 26038272 00010000:00010000:1.0:1789760799.195110:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.195124:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.195134:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.195141:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.195146:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.195156:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.195177:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.195184:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26034176->26038271] (req 26034176->26038271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.195278:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508501888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14907us (15354us total) trans 4294973676 rc 0/0 00000100:00100000:1.0:1789760799.195344:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6388 00000100:00100000:1.0:1789760799.195354:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778261323699 time 778285066645 state 0x2 ref 0 00000100:00100000:0.0:1789760799.197740:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.198724:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508502016 00000100:00100000:1.0:1789760799.198734:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508502016 00000100:00100000:1.0:1789760799.198957:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778261323699 time 778285066645 state 0x0 ref 1 00000100:00100000:1.0:1789760799.198973:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778285327061 time 778285066645 state 0x1 ref 1 00000100:00100000:1.0:1789760799.199005:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778285327061 time 778303715623 state 0x0 ref 1 00000100:00100000:1.0:1789760799.199018:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508502016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.199076:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26038272, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:1.0:1789760799.199085:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.199101:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.199105:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.199111:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.199126:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.218391:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973677 is committed 00002000:00010000:1.0:1789760799.218487:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26038272 -> 26042368 00010000:00010000:1.0:1789760799.218502:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.218517:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.218526:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.218533:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.218538:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.218546:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.218565:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.218572:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26038272->26042367] (req 26038272->26042367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.218671:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508502016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19651us (20930us total) trans 4294973677 rc 0/0 00000100:00100000:1.0:1789760799.218682:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6389 00000100:00100000:1.0:1789760799.218691:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778285327061 time 778303715623 state 0x2 ref 0 00000100:00100000:2.0:1789760799.221278:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.221359:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508502144 00000100:00100000:1.0:1789760799.221369:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508502144 00000100:00100000:1.0:1789760799.221433:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778285327061 time 778303715623 state 0x0 ref 1 00000100:00100000:1.0:1789760799.221447:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778303976039 time 778303715623 state 0x1 ref 1 00000100:00100000:1.0:1789760799.221468:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778303976039 time 778326178583 state 0x0 ref 1 00000100:00100000:1.0:1789760799.221492:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508502144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.221546:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26042368, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:1.0:1789760799.221554:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.221570:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.221575:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.221582:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.221597:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760799.239606:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973678 is committed 00002000:00010000:3.0:1789760799.239730:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26042368 -> 26046464 00010000:00010000:3.0:1789760799.239745:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.239762:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.239771:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.239778:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760799.239782:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.239791:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.239811:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.239818:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe574e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26042368->26046463] (req 26042368->26046463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760799.239905:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508502144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 18411us (18635us total) trans 4294973678 rc 0/0 00000100:00100000:3.0:1789760799.239915:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6390 00000100:00100000:3.0:1789760799.239924:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778303976039 time 778326178583 state 0x2 ref 0 00000100:00100000:0.0:1789760799.242364:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.242458:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508502272 00000100:00100000:1.0:1789760799.242467:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508502272 00000100:00100000:1.0:1789760799.242520:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778303976039 time 778326178583 state 0x0 ref 1 00000100:00100000:1.0:1789760799.242534:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778326438999 time 778326178583 state 0x1 ref 1 00000100:00100000:1.0:1789760799.242558:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778326438999 time 778347269499 state 0x0 ref 1 00000100:00100000:1.0:1789760799.242583:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508502272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.242630:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26046464, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:1.0:1789760799.242638:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.242667:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.242671:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.242678:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.242691:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760799.262336:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973679 is committed 00002000:00010000:1.0:1789760799.262425:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26046464 -> 26050560 00010000:00010000:1.0:1789760799.262441:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.262454:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.262463:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.262470:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.262475:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.262485:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.262504:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.262512:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe574f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26046464->26050559] (req 26046464->26050559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.262599:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508502272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20025us (20238us total) trans 4294973679 rc 0/0 00000100:00100000:1.0:1789760799.262611:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6391 00000100:00100000:1.0:1789760799.262621:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778326438999 time 778347269499 state 0x2 ref 0 00000100:00100000:0.0:1789760799.265465:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.265686:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508502400 00000100:00100000:1.0:1789760799.265697:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508502400 00000100:00100000:1.0:1789760799.265754:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778326438999 time 778347269499 state 0x0 ref 1 00000100:00100000:1.0:1789760799.265770:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778347529915 time 778347269499 state 0x1 ref 1 00000100:00100000:1.0:1789760799.265799:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778347529915 time 778370509348 state 0x0 ref 1 00000100:00100000:1.0:1789760799.265811:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508502400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.265865:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26050560, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:1.0:1789760799.265874:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe574f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.265891:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.265895:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.265902:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe574f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.265916:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe574f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.288975:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973680 is committed 00002000:00010000:1.0:1789760799.289054:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26050560 -> 26054656 00010000:00010000:1.0:1789760799.289069:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe574f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.289082:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe574f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.289091:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe574f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.289099:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.289104:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe574f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.289113:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe574f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.289133:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe574f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.289142:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe574f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26050560->26054655] (req 26050560->26054655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.289263:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508502400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23450us (23805us total) trans 4294973680 rc 0/0 00000100:00100000:1.0:1789760799.289274:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6392 00000100:00100000:1.0:1789760799.289325:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778347529915 time 778370509348 state 0x2 ref 0 00000100:00100000:2.0:1789760799.292963:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760799.293049:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508502528 00000100:00100000:3.0:1789760799.293058:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508502528 00000100:00100000:3.0:1789760799.293110:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778347529915 time 778370509348 state 0x0 ref 1 00000100:00100000:3.0:1789760799.293134:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778370769764 time 778370509348 state 0x1 ref 1 00000100:00100000:3.0:1789760799.293157:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778370769764 time 778397868648 state 0x0 ref 1 00000100:00100000:3.0:1789760799.293167:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508502528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760799.293282:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26054656, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760799.293337:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe574fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.293353:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760799.293357:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760799.293363:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe574fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.293376:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe574fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.312078:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973681 is committed 00002000:00010000:3.0:1789760799.312162:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26054656 -> 26058752 00010000:00010000:3.0:1789760799.312175:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe574fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.312184:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe574fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.312194:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe574fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.312199:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760799.312202:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe574fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.312207:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe574fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.312222:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe574fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.312226:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe574fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26054656->26058751] (req 26054656->26058751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760799.312336:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508502528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19165us (19379us total) trans 4294973681 rc 0/0 00000100:00100000:3.0:1789760799.312345:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6393 00000100:00100000:3.0:1789760799.312354:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778370769764 time 778397868648 state 0x2 ref 0 00000100:00100000:0.0:1789760799.315502:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.315701:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508502656 00000100:00100000:1.0:1789760799.315711:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508502656 00000100:00100000:1.0:1789760799.315768:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778370769764 time 778397868648 state 0x0 ref 1 00000100:00100000:1.0:1789760799.315783:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778398129064 time 778397868648 state 0x1 ref 1 00000100:00100000:1.0:1789760799.315820:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778398129064 time 778420529827 state 0x0 ref 1 00000100:00100000:1.0:1789760799.315833:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508502656:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.315903:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26058752, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:1.0:1789760799.315912:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57505 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.315928:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.315932:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.315939:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57505 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.315959:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57505 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760799.339158:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26058752 -> 26062848 00000001:00080000:2.0:1789760799.339166:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973682 is committed 00010000:00010000:0.0:1789760799.339178:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57505 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.339192:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57505 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.339201:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57505 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.339209:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760799.339214:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57505 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.339223:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57505 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.339243:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57505 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.339251:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57505 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26058752->26062847] (req 26058752->26062847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760799.339382:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508502656:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23545us (23882us total) trans 4294973682 rc 0/0 00000100:00100000:0.0:1789760799.339393:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6394 00000100:00100000:0.0:1789760799.339402:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778398129064 time 778420529827 state 0x2 ref 0 00000100:00100000:0.0:1789760799.343890:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.343983:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508502784 00000100:00100000:1.0:1789760799.343992:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508502784 00000100:00100000:1.0:1789760799.344043:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778398129064 time 778420529827 state 0x0 ref 1 00000100:00100000:1.0:1789760799.344059:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778420790243 time 778420529827 state 0x1 ref 1 00000100:00100000:1.0:1789760799.344091:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778420790243 time 778448801869 state 0x0 ref 1 00000100:00100000:1.0:1789760799.344104:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508502784:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.344149:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26062848, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:1.0:1789760799.344158:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5750c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.344174:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.344178:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.344185:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5750c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.344200:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5750c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760799.381357:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26062848 -> 26066944 00000001:00080000:3.0:1789760799.381366:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973683 is committed 00010000:00010000:2.0:1789760799.381377:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5750c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.381391:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5750c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.381400:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5750c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.381407:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760799.381412:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5750c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.381422:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5750c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.381441:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5750c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.381454:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5750c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26062848->26066943] (req 26062848->26066943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760799.381540:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508502784:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37434us (37656us total) trans 4294973683 rc 0/0 00000100:00100000:2.0:1789760799.381551:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6395 00000100:00100000:2.0:1789760799.381562:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778420790243 time 778448801869 state 0x2 ref 0 00000100:00100000:2.0:1789760799.389358:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760799.389453:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508502912 00000100:00100000:3.0:1789760799.389470:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508502912 00000100:00100000:3.0:1789760799.389550:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778420790243 time 778448801869 state 0x0 ref 1 00000100:00100000:3.0:1789760799.389584:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778449062285 time 778448801869 state 0x1 ref 1 00000100:00100000:3.0:1789760799.389610:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778449062285 time 778494320387 state 0x0 ref 1 00000100:00100000:3.0:1789760799.389625:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508502912:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760799.389702:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26066944, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:3.0:1789760799.389714:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57513 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.389733:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760799.389738:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760799.389746:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57513 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.389763:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57513 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760799.434106:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973684 is committed 00002000:00010000:3.0:1789760799.434198:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26066944 -> 26071040 00010000:00010000:3.0:1789760799.434218:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57513 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.434234:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57513 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.434244:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57513 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.434255:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760799.434260:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57513 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.434267:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57513 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.434307:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57513 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.434313:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57513 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26066944->26071039] (req 26066944->26071039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760799.434457:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508502912:12345-192.168.203.23@tcp:4:dd.0 Request processed in 44830us (45101us total) trans 4294973684 rc 0/0 00000100:00100000:3.0:1789760799.434469:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6396 00000100:00100000:3.0:1789760799.434481:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778449062285 time 778494320387 state 0x2 ref 0 00000100:00100000:0.0:1789760799.439782:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.439890:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508503040 00000100:00100000:1.0:1789760799.439900:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508503040 00000100:00100000:1.0:1789760799.440024:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778449062285 time 778494320387 state 0x0 ref 1 00000100:00100000:1.0:1789760799.440041:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778494580803 time 778494320387 state 0x1 ref 1 00000100:00100000:1.0:1789760799.440079:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778494580803 time 778544789259 state 0x0 ref 1 00000100:00100000:1.0:1789760799.440094:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508503040:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.440849:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26071040, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:1.0:1789760799.440860:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5751a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.440876:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.440880:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.440886:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5751a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.440899:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5751a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.465278:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973685 is committed 00002000:00010000:1.0:1789760799.465553:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26071040 -> 26075136 00010000:00010000:1.0:1789760799.465569:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5751a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.465581:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5751a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.465595:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5751a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.465601:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.465606:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5751a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.465613:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5751a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.465632:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5751a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.465652:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5751a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26071040->26075135] (req 26071040->26075135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.465781:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508503040:12345-192.168.203.23@tcp:4:dd.0 Request processed in 25685us (26003us total) trans 4294973685 rc 0/0 00000100:00100000:1.0:1789760799.465791:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6397 00000100:00100000:1.0:1789760799.465800:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778494580803 time 778544789259 state 0x2 ref 0 00000100:00100000:0.0:1789760799.469512:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760799.470024:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508503168 00000100:00100000:2.0:1789760799.470032:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508503168 00000100:00100000:2.0:1789760799.470086:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778494580803 time 778544789259 state 0x0 ref 1 00000100:00100000:2.0:1789760799.470101:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778545049675 time 778544789259 state 0x1 ref 1 00000100:00100000:2.0:1789760799.470132:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778545049675 time 778574842547 state 0x0 ref 1 00000100:00100000:2.0:1789760799.470143:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508503168:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760799.470189:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26075136, mtime: 0x6aad951f, atime: 0x0, ctime: 0x6aad951f, blocks: 0xc807 00010000:00010000:2.0:1789760799.470198:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe57521 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.470213:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:2.0:1789760799.470217:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760799.470224:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe57521 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760799.470237:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe57521 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.487391:0:6454:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760799.507245:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973686 is committed 00002000:00010000:1.0:1789760799.507390:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26075136 -> 26079232 00002000:00010000:1.0:1789760799.507395:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760799 -> 1789760800 00002000:00010000:1.0:1789760799.507398:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760799 -> 1789760800 00010000:00010000:1.0:1789760799.507411:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe57521 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.507424:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe57521 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.507432:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe57521 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.507436:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.507441:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe57521 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.507449:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe57521 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.507468:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe57521 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.507475:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe57521 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26075136->26079231] (req 26075136->26079231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.507557:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508503168:12345-192.168.203.23@tcp:4:dd.0 Request processed in 37410us (38047us total) trans 4294973686 rc 0/0 00000100:00100000:1.0:1789760799.507567:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6398 00000100:00100000:1.0:1789760799.507576:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778545049675 time 778574842547 state 0x2 ref 0 00000100:00100000:2.0:1789760799.515789:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760799.515854:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508503296 00000100:00100000:3.0:1789760799.515863:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508503296 00000100:00100000:3.0:1789760799.516056:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778545049675 time 778574842547 state 0x0 ref 1 00000100:00100000:3.0:1789760799.516073:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778575102963 time 778574842547 state 0x1 ref 1 00000100:00100000:3.0:1789760799.516100:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778575102963 time 778620810606 state 0x0 ref 1 00000100:00100000:3.0:1789760799.516117:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508503296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760799.516173:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26079232, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc807 00010000:00010000:3.0:1789760799.516182:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57528 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.516198:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760799.516203:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760799.516209:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57528 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.516372:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57528 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.543332:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973687 is committed 00002000:00010000:1.0:1789760799.543387:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26079232 -> 26083328 00010000:00010000:1.0:1789760799.543403:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57528 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.543415:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57528 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.543425:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57528 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.543433:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.543437:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57528 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.543447:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57528 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.543467:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57528 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.543475:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe57528 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26079232->26083327] (req 26079232->26083327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.543571:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508503296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27452us (27785us total) trans 4294973687 rc 0/0 00000100:00100000:1.0:1789760799.543581:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6399 00000100:00100000:1.0:1789760799.543591:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778575102963 time 778620810606 state 0x2 ref 0 00000100:00100000:0.0:1789760799.551826:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.552254:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508503424 00000100:00100000:1.0:1789760799.552264:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508503424 00000100:00100000:1.0:1789760799.552394:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778575102963 time 778620810606 state 0x0 ref 1 00000100:00100000:1.0:1789760799.552409:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778621071022 time 778620810606 state 0x1 ref 1 00000100:00100000:1.0:1789760799.552437:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778621071022 time 778657148168 state 0x0 ref 1 00000100:00100000:1.0:1789760799.552450:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508503424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.552496:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26083328, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc807 00010000:00010000:1.0:1789760799.552504:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5752f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.552520:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.552524:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.552530:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5752f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.552544:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5752f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.579193:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973688 is committed 00002000:00010000:1.0:1789760799.579563:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26083328 -> 26087424 00010000:00010000:1.0:1789760799.579590:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5752f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.579611:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5752f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.579620:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5752f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.579626:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.579631:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5752f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.579649:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5752f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.579669:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5752f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.579676:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5752f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26083328->26087423] (req 26083328->26087423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.579776:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508503424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 27324us (27957us total) trans 4294973688 rc 0/0 00000100:00100000:1.0:1789760799.579786:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6400 00000100:00100000:1.0:1789760799.579796:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778621071022 time 778657148168 state 0x2 ref 0 00000100:00100000:0.0:1789760799.583113:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.583236:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508503552 00000100:00100000:1.0:1789760799.583245:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508503552 00000100:00100000:1.0:1789760799.583322:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778621071022 time 778657148168 state 0x0 ref 1 00000100:00100000:1.0:1789760799.583333:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778657408584 time 778657148168 state 0x1 ref 1 00000100:00100000:1.0:1789760799.583359:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778657408584 time 778688071316 state 0x0 ref 1 00000100:00100000:1.0:1789760799.583368:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508503552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.583405:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26087424, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc807 00010000:00010000:1.0:1789760799.583411:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57536 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.583422:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.583432:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.583437:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57536 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.583449:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57536 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760799.612144:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26087424 -> 26091520 00010000:00010000:1.0:1789760799.612163:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57536 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.612177:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57536 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.612193:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57536 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.612201:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.612206:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57536 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.612216:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57536 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.612237:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57536 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.612245:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57536 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26087424->26091519] (req 26087424->26091519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.612368:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508503552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 28996us (29260us total) trans 4294973689 rc 0/0 00000100:00100000:1.0:1789760799.612380:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6401 00000100:00100000:1.0:1789760799.612389:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778657408584 time 778688071316 state 0x2 ref 0 00000001:00080000:0.0:1789760799.612547:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973689 is committed 00000100:00100000:2.0:1789760799.615453:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.616118:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508503680 00000100:00100000:1.0:1789760799.616129:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508503680 00000100:00100000:1.0:1789760799.616175:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778657408584 time 778688071316 state 0x0 ref 1 00000100:00100000:1.0:1789760799.616194:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778688331732 time 778688071316 state 0x1 ref 1 00000100:00100000:1.0:1789760799.616258:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778688331732 time 778720967163 state 0x0 ref 1 00000100:00100000:1.0:1789760799.616270:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508503680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.616340:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26091520, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc807 00010000:00010000:1.0:1789760799.616349:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5753d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.616363:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.616367:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.616374:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5753d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.616387:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5753d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.640568:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973690 is committed 00002000:00010000:1.0:1789760799.640636:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26091520 -> 26095616 00010000:00010000:1.0:1789760799.640663:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5753d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.640674:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5753d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.640680:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5753d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.640684:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.640688:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5753d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.640695:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5753d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.640716:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5753d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.640720:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5753d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26091520->26095615] (req 26091520->26095615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.640839:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508503680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24564us (25387us total) trans 4294973690 rc 0/0 00000100:00100000:1.0:1789760799.640848:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6402 00000100:00100000:1.0:1789760799.640857:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778688331732 time 778720967163 state 0x2 ref 0 00000100:00100000:0.0:1789760799.643904:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.643973:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508503808 00000100:00100000:1.0:1789760799.643982:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508503808 00000100:00100000:1.0:1789760799.644032:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778688331732 time 778720967163 state 0x0 ref 1 00000100:00100000:1.0:1789760799.644046:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778721227579 time 778720967163 state 0x1 ref 1 00000100:00100000:1.0:1789760799.644117:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778721227579 time 778748827596 state 0x0 ref 1 00000100:00100000:1.0:1789760799.644129:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508503808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.644175:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26095616, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc807 00010000:00010000:1.0:1789760799.644184:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57544 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.644198:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.644202:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.644208:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57544 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.644220:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57544 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760799.663134:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973691 is committed 00002000:00010000:3.0:1789760799.663463:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26095616 -> 26099712 00010000:00010000:3.0:1789760799.663478:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57544 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.663490:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57544 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.663499:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57544 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.663505:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760799.663510:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57544 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.663520:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57544 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.663540:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57544 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.663548:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57544 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26095616->26099711] (req 26095616->26099711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760799.663634:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508503808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19502us (19734us total) trans 4294973691 rc 0/0 00000100:00100000:3.0:1789760799.663653:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6403 00000100:00100000:3.0:1789760799.663662:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778721227579 time 778748827596 state 0x2 ref 0 00000100:00100000:3.0:1789760799.668327:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.668426:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508503936 00000100:00100000:1.0:1789760799.668436:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508503936 00000100:00100000:0.0:1789760799.668802:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778721227579 time 778748827596 state 0x0 ref 1 00000100:00100000:0.0:1789760799.668816:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778749088012 time 778748827596 state 0x1 ref 1 00000100:00100000:0.0:1789760799.668847:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778749088012 time 778773558067 state 0x0 ref 1 00000100:00100000:0.0:1789760799.668860:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508503936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760799.668903:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26099712, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc807 00010000:00010000:0.0:1789760799.668912:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5754b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.668927:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:0.0:1789760799.668932:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760799.668939:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5754b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760799.668952:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5754b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760799.703516:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973692 is committed 00002000:00010000:1.0:1789760799.703618:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26099712 -> 26103808 00002000:00010000:1.0:1789760799.703624:0:14639:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 51207 -> 51209 00010000:00010000:1.0:1789760799.703649:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5754b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.703665:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5754b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.703676:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5754b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.703693:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.703703:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5754b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.703714:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5754b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.703750:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5754b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.703759:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5754b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26099712->26103807] (req 26099712->26103807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.703882:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508503936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 35019us (35592us total) trans 4294973692 rc 0/0 00000100:00100000:1.0:1789760799.703893:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6404 00000100:00100000:1.0:1789760799.703904:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778749088012 time 778773558067 state 0x2 ref 0 00000100:00100000:2.0:1789760799.710831:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.710928:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508504064 00000100:00100000:1.0:1789760799.710940:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508504064 00000100:00100000:1.0:1789760799.711004:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778749088012 time 778773558067 state 0x0 ref 1 00000100:00100000:1.0:1789760799.711022:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778773818483 time 778773558067 state 0x1 ref 1 00000100:00100000:1.0:1789760799.711092:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778773818483 time 778815794595 state 0x0 ref 1 00000100:00100000:1.0:1789760799.711105:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508504064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.711162:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26103808, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760799.711172:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57552 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.711192:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.711197:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.711204:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57552 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.711220:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57552 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760799.735602:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973693 is committed 00002000:00010000:1.0:1789760799.735748:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26103808 -> 26107904 00010000:00010000:1.0:1789760799.735769:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57552 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.735792:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57552 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.735803:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57552 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.735811:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.735816:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57552 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.735826:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57552 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.735851:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57552 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.735861:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57552 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26103808->26107903] (req 26103808->26107903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.735989:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508504064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 24878us (25164us total) trans 4294973693 rc 0/0 00000100:00100000:1.0:1789760799.736002:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6405 00000100:00100000:1.0:1789760799.736014:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778773818483 time 778815794595 state 0x2 ref 0 00000100:00100000:0.0:1789760799.738807:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760799.738936:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508504192 00000100:00100000:3.0:1789760799.738950:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508504192 00000100:00100000:3.0:1789760799.739014:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778773818483 time 778815794595 state 0x0 ref 1 00000100:00100000:3.0:1789760799.739028:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778816055011 time 778815794595 state 0x1 ref 1 00000100:00100000:3.0:1789760799.739066:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778816055011 time 778843776466 state 0x0 ref 1 00000100:00100000:3.0:1789760799.739081:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508504192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760799.739138:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26107904, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760799.739147:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe57559 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.739172:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:3.0:1789760799.739177:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760799.739184:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe57559 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.739202:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe57559 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760799.758870:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973694 is committed 00002000:00010000:1.0:1789760799.758889:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26107904 -> 26112000 00010000:00010000:1.0:1789760799.758909:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe57559 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.758930:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe57559 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.758943:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe57559 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.758951:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.758956:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe57559 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.758966:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe57559 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.758997:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe57559 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.759006:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe57559 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26107904->26111999] (req 26107904->26111999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.759166:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508504192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20073us (20360us total) trans 4294973694 rc 0/0 00000100:00100000:1.0:1789760799.759179:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6406 00000100:00100000:1.0:1789760799.759191:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778816055011 time 778843776466 state 0x2 ref 0 00000100:00100000:3.0:1789760799.761326:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.761564:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508504320 00000100:00100000:1.0:1789760799.761576:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508504320 00000100:00100000:1.0:1789760799.761665:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778816055011 time 778843776466 state 0x0 ref 1 00000100:00100000:1.0:1789760799.761683:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778844036882 time 778843776466 state 0x1 ref 1 00000100:00100000:1.0:1789760799.761720:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778844036882 time 778866429501 state 0x0 ref 1 00000100:00100000:1.0:1789760799.761734:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508504320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.761794:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26112000, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760799.761805:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57560 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.761823:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.761829:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.761837:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57560 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.761853:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57560 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760799.782800:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973695 is committed 00002000:00010000:1.0:1789760799.782891:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26112000 -> 26116096 00010000:00010000:1.0:1789760799.782908:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57560 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.782919:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57560 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.782939:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57560 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.782947:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.782952:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57560 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.782962:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57560 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.782988:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57560 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.783000:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57560 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26112000->26116095] (req 26112000->26116095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.783110:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508504320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 21372us (21785us total) trans 4294973695 rc 0/0 00000100:00100000:1.0:1789760799.783123:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6407 00000100:00100000:1.0:1789760799.783135:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778844036882 time 778866429501 state 0x2 ref 0 00000100:00100000:2.0:1789760799.785812:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.785923:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508504448 00000100:00100000:1.0:1789760799.785936:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508504448 00000100:00100000:1.0:1789760799.786001:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778844036882 time 778866429501 state 0x0 ref 1 00000100:00100000:1.0:1789760799.786020:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778866689917 time 778866429501 state 0x1 ref 1 00000100:00100000:1.0:1789760799.786061:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778866689917 time 778890771341 state 0x0 ref 1 00000100:00100000:1.0:1789760799.786076:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508504448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.786133:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26116096, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760799.786143:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57567 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.786162:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.786167:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.786174:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57567 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.786206:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57567 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760799.805554:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973696 is committed 00002000:00010000:1.0:1789760799.805592:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26116096 -> 26120192 00010000:00010000:1.0:1789760799.805617:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57567 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.805633:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57567 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.805652:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57567 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.805661:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.805667:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57567 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.805686:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57567 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.805713:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57567 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.805721:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57567 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26116096->26120191] (req 26116096->26120191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.805847:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508504448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19766us (20038us total) trans 4294973696 rc 0/0 00000100:00100000:1.0:1789760799.805858:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6408 00000100:00100000:1.0:1789760799.805867:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778866689917 time 778890771341 state 0x2 ref 0 00000100:00100000:0.0:1789760799.808044:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.808215:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508504576 00000100:00100000:1.0:1789760799.808228:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508504576 00000100:00100000:1.0:1789760799.808322:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778866689917 time 778890771341 state 0x0 ref 1 00000100:00100000:1.0:1789760799.808342:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778891031757 time 778890771341 state 0x1 ref 1 00000100:00100000:1.0:1789760799.808374:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778891031757 time 778913084588 state 0x0 ref 1 00000100:00100000:1.0:1789760799.808388:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508504576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.808453:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26120192, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760799.808462:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5756e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.808482:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.808487:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.808495:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5756e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.808511:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5756e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760799.831230:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973697 is committed 00002000:00010000:3.0:1789760799.831793:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26120192 -> 26124288 00010000:00010000:3.0:1789760799.831815:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5756e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.831831:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5756e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.831841:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5756e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.831853:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760799.831859:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5756e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.831870:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5756e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.831893:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5756e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.831901:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe5756e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26120192->26124287] (req 26120192->26124287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760799.832020:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508504576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 23628us (23980us total) trans 4294973697 rc 0/0 00000100:00100000:3.0:1789760799.832031:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6409 00000100:00100000:3.0:1789760799.832043:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778891031757 time 778913084588 state 0x2 ref 0 00000100:00100000:3.0:1789760799.834940:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.835181:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508504704 00000100:00100000:1.0:1789760799.835195:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508504704 00000100:00100000:1.0:1789760799.835266:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778891031757 time 778913084588 state 0x0 ref 1 00000100:00100000:1.0:1789760799.835328:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778913345004 time 778913084588 state 0x1 ref 1 00000100:00100000:1.0:1789760799.835353:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778913345004 time 778940063881 state 0x0 ref 1 00000100:00100000:1.0:1789760799.835368:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508504704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.835431:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26124288, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760799.835441:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57575 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.835459:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.835464:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.835473:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57575 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.835489:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57575 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760799.868775:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973698 is committed 00002000:00010000:3.0:1789760799.869035:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26124288 -> 26128384 00010000:00010000:3.0:1789760799.869055:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57575 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.869066:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57575 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.869072:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57575 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.869081:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760799.869085:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57575 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.869091:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57575 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.869110:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57575 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.869114:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe57575 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26124288->26128383] (req 26124288->26128383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760799.869210:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508504704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 33839us (34275us total) trans 4294973698 rc 0/0 00000100:00100000:3.0:1789760799.869222:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6410 00000100:00100000:3.0:1789760799.869234:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778913345004 time 778940063881 state 0x2 ref 0 00000100:00100000:2.0:1789760799.872928:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760799.873094:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508504832 00000100:00100000:1.0:1789760799.873106:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508504832 00000100:00100000:1.0:1789760799.873172:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778913345004 time 778940063881 state 0x0 ref 1 00000100:00100000:1.0:1789760799.873191:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778940324297 time 778940063881 state 0x1 ref 1 00000100:00100000:1.0:1789760799.873257:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778940324297 time 778977967090 state 0x0 ref 1 00000100:00100000:1.0:1789760799.873271:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508504832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760799.873379:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26128384, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760799.873391:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5757c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.873410:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=778s v=(0 0) 00010000:00010000:1.0:1789760799.873416:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760799.873424:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5757c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.873453:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5757c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.902802:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973699 is committed 00002000:00010000:3.0:1789760799.902845:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26128384 -> 26132480 00010000:00010000:3.0:1789760799.902862:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5757c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.902875:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5757c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.902890:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5757c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.902908:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760799.902916:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5757c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.902926:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5757c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.902952:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5757c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.902960:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000050769f6b/0xb49177c0efe5757c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26128384->26132479] (req 26128384->26132479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760799.903055:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508504832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 29781us (30133us total) trans 4294973699 rc 0/0 00000100:00100000:3.0:1789760799.903065:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6411 00000100:00100000:3.0:1789760799.903075:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778940324297 time 778977967090 state 0x2 ref 0 00000100:00100000:0.0:1789760799.914002:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760799.914093:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508504960 00000100:00100000:3.0:1789760799.914103:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508504960 00000100:00100000:3.0:1789760799.914155:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778940324297 time 778977967090 state 0x0 ref 1 00000100:00100000:3.0:1789760799.914171:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778978227506 time 778977967090 state 0x1 ref 1 00000100:00100000:3.0:1789760799.914228:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778978227506 time 779018938948 state 0x0 ref 1 00000100:00100000:3.0:1789760799.914239:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c1fd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508504960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760799.914314:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26132480, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760799.914323:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe57583 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.914343:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760799.914348:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760799.914354:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe57583 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.914368:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe57583 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.928946:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973700 is committed 00002000:00010000:3.0:1789760799.929099:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26132480 -> 26136576 00010000:00010000:3.0:1789760799.929119:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe57583 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.929132:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe57583 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.929159:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe57583 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.929170:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760799.929176:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe57583 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.929189:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe57583 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.929208:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe57583 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.929215:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe57583 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26132480->26136575] (req 26132480->26136575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760799.929336:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c1fd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508504960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15094us (15340us total) trans 4294973700 rc 0/0 00000100:00100000:3.0:1789760799.929348:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6412 00000100:00100000:3.0:1789760799.929359:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778978227506 time 779018938948 state 0x2 ref 0 00000100:00100000:0.0:1789760799.932812:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760799.932892:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508505088 00000100:00100000:3.0:1789760799.932902:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508505088 00000100:00100000:3.0:1789760799.932946:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 778978227506 time 779018938948 state 0x0 ref 1 00000100:00100000:3.0:1789760799.932957:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779019199364 time 779018938948 state 0x1 ref 1 00000100:00100000:3.0:1789760799.932977:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779019199364 time 779037688784 state 0x0 ref 1 00000100:00100000:3.0:1789760799.932998:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508505088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760799.933042:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26136576, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760799.933051:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5758a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.933064:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760799.933068:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760799.933072:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5758a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.933082:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5758a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760799.948885:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973701 is committed 00002000:00010000:3.0:1789760799.948967:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26136576 -> 26140672 00010000:00010000:3.0:1789760799.948983:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5758a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.948996:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5758a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.949006:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5758a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.949014:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760799.949018:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5758a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.949027:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5758a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.949053:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5758a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.949061:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe5758a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26136576->26140671] (req 26136576->26140671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760799.949160:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508505088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16169us (16352us total) trans 4294973701 rc 0/0 00000100:00100000:3.0:1789760799.949169:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6413 00000100:00100000:3.0:1789760799.949179:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779019199364 time 779037688784 state 0x2 ref 0 00000100:00100000:2.0:1789760799.954605:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760799.954699:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508505216 00000100:00100000:3.0:1789760799.954709:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508505216 00000100:00100000:3.0:1789760799.954761:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779019199364 time 779037688784 state 0x0 ref 1 00000100:00100000:3.0:1789760799.954773:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779037949200 time 779037688784 state 0x1 ref 1 00000100:00100000:3.0:1789760799.954796:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779037949200 time 779059506058 state 0x0 ref 1 00000100:00100000:3.0:1789760799.954808:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508505216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760799.954852:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26140672, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760799.954859:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57591 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.954871:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760799.954874:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760799.954878:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57591 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.954888:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57591 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760799.967657:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973702 is committed 00002000:00010000:3.0:1789760799.967702:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26140672 -> 26144768 00010000:00010000:3.0:1789760799.967717:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57591 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.967730:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57591 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.967740:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57591 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.967747:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760799.967751:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57591 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.967760:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57591 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.967779:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57591 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.967786:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57591 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26140672->26144767] (req 26140672->26144767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760799.967875:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508505216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13065us (13273us total) trans 4294973702 rc 0/0 00000100:00100000:3.0:1789760799.967885:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6414 00000100:00100000:3.0:1789760799.967895:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779037949200 time 779059506058 state 0x2 ref 0 00000100:00100000:0.0:1789760799.974381:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760799.974478:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508505344 00000100:00100000:3.0:1789760799.974489:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508505344 00000100:00100000:3.0:1789760799.974541:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779037949200 time 779059506058 state 0x0 ref 1 00000100:00100000:3.0:1789760799.974558:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779059766474 time 779059506058 state 0x1 ref 1 00000100:00100000:3.0:1789760799.974590:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779059766474 time 779079300483 state 0x0 ref 1 00000100:00100000:3.0:1789760799.974603:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508505344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760799.974662:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26144768, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760799.974672:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57598 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.974688:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760799.974692:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760799.974699:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57598 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.974712:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57598 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760799.988687:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973703 is committed 00002000:00010000:1.0:1789760799.988721:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26144768 -> 26148864 00010000:00010000:1.0:1789760799.988733:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57598 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.988745:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57598 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.988764:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57598 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.988772:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760799.988776:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57598 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.988786:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57598 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.988806:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57598 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760799.988814:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe57598 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26144768->26148863] (req 26144768->26148863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760799.988906:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508505344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14301us (14530us total) trans 4294973703 rc 0/0 00000100:00100000:1.0:1789760799.988916:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6415 00000100:00100000:1.0:1789760799.988926:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779059766474 time 779079300483 state 0x2 ref 0 00000100:00100000:0.0:1789760799.991032:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760799.991136:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508505472 00000100:00100000:3.0:1789760799.991145:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508505472 00000100:00100000:3.0:1789760799.991194:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779059766474 time 779079300483 state 0x0 ref 1 00000100:00100000:3.0:1789760799.991209:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779079560899 time 779079300483 state 0x1 ref 1 00000100:00100000:3.0:1789760799.991235:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779079560899 time 779095945779 state 0x0 ref 1 00000100:00100000:3.0:1789760799.991248:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508505472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760799.991320:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26148864, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760799.991329:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5759f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.991356:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760799.991360:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760799.991366:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5759f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760799.991380:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5759f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760800.010704:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973704 is committed 00002000:00010000:1.0:1789760800.010803:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26148864 -> 26152960 00010000:00010000:1.0:1789760800.010817:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5759f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.010836:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5759f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.010842:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5759f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.010847:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.010850:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5759f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.010856:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5759f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.010872:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5759f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.010884:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe5759f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26148864->26152959] (req 26148864->26152959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.011089:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508505472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19839us (20059us total) trans 4294973704 rc 0/0 00000100:00100000:1.0:1789760800.011100:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6416 00000100:00100000:1.0:1789760800.011110:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779079560899 time 779095945779 state 0x2 ref 0 00000100:00100000:2.0:1789760800.016546:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760800.016653:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508505600 00000100:00100000:3.0:1789760800.016663:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508505600 00000100:00100000:3.0:1789760800.016782:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779079560899 time 779095945779 state 0x0 ref 1 00000100:00100000:3.0:1789760800.016801:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779096206195 time 779095945779 state 0x1 ref 1 00000100:00100000:3.0:1789760800.016834:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779096206195 time 779121544014 state 0x0 ref 1 00000100:00100000:3.0:1789760800.016847:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508505600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760800.016893:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26152960, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760800.016902:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.016917:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760800.016921:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760800.016927:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.016942:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760800.035097:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26152960 -> 26157056 00010000:00010000:3.0:1789760800.035129:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.035143:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.035153:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.035165:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.035170:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.035179:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.035199:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.035208:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26152960->26157055] (req 26152960->26157055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.035378:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508505600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 18528us (18837us total) trans 4294973705 rc 0/0 00000100:00100000:3.0:1789760800.035389:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6417 00000100:00100000:3.0:1789760800.035398:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779096206195 time 779121544014 state 0x2 ref 0 00000001:00080000:3.0:1789760800.035453:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973705 is committed 00000100:00100000:0.0:1789760800.038032:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.038152:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508505728 00000100:00100000:1.0:1789760800.038162:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508505728 00000100:00100000:1.0:1789760800.038213:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779096206195 time 779121544014 state 0x0 ref 1 00000100:00100000:1.0:1789760800.038228:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779121804430 time 779121544014 state 0x1 ref 1 00000100:00100000:1.0:1789760800.038319:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779121804430 time 779143028043 state 0x0 ref 1 00000100:00100000:1.0:1789760800.038333:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508505728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.038392:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26157056, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760800.038401:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.038421:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.038425:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.038431:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.038446:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.054739:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973706 is committed 00002000:00010000:1.0:1789760800.054792:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26157056 -> 26161152 00010000:00010000:1.0:1789760800.054807:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.054820:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.054834:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.054840:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.054845:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.054852:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.054872:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.054879:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26157056->26161151] (req 26157056->26161151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.054962:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508505728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16628us (16935us total) trans 4294973706 rc 0/0 00000100:00100000:1.0:1789760800.054971:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6418 00000100:00100000:1.0:1789760800.054980:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779121804430 time 779143028043 state 0x2 ref 0 00000100:00100000:0.0:1789760800.058975:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.059032:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508505856 00000100:00100000:1.0:1789760800.059042:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508505856 00000100:00100000:1.0:1789760800.059095:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779121804430 time 779143028043 state 0x0 ref 1 00000100:00100000:1.0:1789760800.059110:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779143288459 time 779143028043 state 0x1 ref 1 00000100:00100000:1.0:1789760800.059137:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779143288459 time 779163848382 state 0x0 ref 1 00000100:00100000:1.0:1789760800.059149:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508505856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.059197:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26161152, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760800.059207:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe575b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.059222:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.059226:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.059233:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe575b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.059247:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe575b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760800.074890:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973707 is committed 00002000:00010000:3.0:1789760800.075004:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26161152 -> 26165248 00010000:00010000:3.0:1789760800.075019:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe575b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.075033:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe575b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.075048:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe575b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.075056:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.075060:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe575b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.075067:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe575b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.075082:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe575b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.075087:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e614dfd/0xb49177c0efe575b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26161152->26165247] (req 26161152->26165247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.075174:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04886362680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508505856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16023us (16203us total) trans 4294973707 rc 0/0 00000100:00100000:3.0:1789760800.075184:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6419 00000100:00100000:3.0:1789760800.075193:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779143288459 time 779163848382 state 0x2 ref 0 00000100:00100000:2.0:1789760800.080272:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760800.080335:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508505984 00000100:00100000:3.0:1789760800.080345:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508505984 00000100:00100000:3.0:1789760800.080399:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779143288459 time 779163848382 state 0x0 ref 1 00000100:00100000:3.0:1789760800.080414:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779164108798 time 779163848382 state 0x1 ref 1 00000100:00100000:3.0:1789760800.080442:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779164108798 time 779185152408 state 0x0 ref 1 00000100:00100000:3.0:1789760800.080455:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508505984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760800.080501:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26165248, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760800.080511:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.080526:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760800.080531:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760800.080537:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.080549:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760800.102587:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973708 is committed 00002000:00010000:3.0:1789760800.102884:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26165248 -> 26169344 00010000:00010000:3.0:1789760800.102905:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.102917:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.102926:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.102934:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.102939:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.102948:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.102969:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.102977:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26165248->26169343] (req 26165248->26169343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.103079:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508505984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22622us (22813us total) trans 4294973708 rc 0/0 00000100:00100000:3.0:1789760800.103091:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6420 00000100:00100000:3.0:1789760800.103109:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779164108798 time 779185152408 state 0x2 ref 0 00000100:00100000:0.0:1789760800.105477:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.105618:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508506112 00000100:00100000:1.0:1789760800.105630:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508506112 00000100:00100000:1.0:1789760800.105709:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779164108798 time 779185152408 state 0x0 ref 1 00000100:00100000:1.0:1789760800.105727:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779185412824 time 779185152408 state 0x1 ref 1 00000100:00100000:1.0:1789760800.105761:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779185412824 time 779210470829 state 0x0 ref 1 00000100:00100000:1.0:1789760800.105774:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508506112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.105835:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26169344, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760800.105844:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.105864:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.105869:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.105877:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.105891:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760800.124438:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973709 is committed 00002000:00010000:3.0:1789760800.124621:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26169344 -> 26173440 00010000:00010000:3.0:1789760800.124645:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.124659:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.124668:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.124681:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.124691:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.124700:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.124719:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.124727:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe575c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26169344->26173439] (req 26169344->26173439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.124822:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508506112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19046us (19351us total) trans 4294973709 rc 0/0 00000100:00100000:3.0:1789760800.124832:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6421 00000100:00100000:3.0:1789760800.124841:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779185412824 time 779210470829 state 0x2 ref 0 00000100:00100000:0.0:1789760800.126914:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760800.126954:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508506240 00000100:00100000:3.0:1789760800.126964:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508506240 00000100:00100000:3.0:1789760800.127011:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779185412824 time 779210470829 state 0x0 ref 1 00000100:00100000:3.0:1789760800.127025:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779210731245 time 779210470829 state 0x1 ref 1 00000100:00100000:3.0:1789760800.127049:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779210731245 time 779231759188 state 0x0 ref 1 00000100:00100000:3.0:1789760800.127066:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca1ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508506240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760800.127109:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26173440, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760800.127178:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.127193:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760800.127197:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760800.127204:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.127218:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.139700:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973710 is committed 00002000:00010000:3.0:1789760800.139715:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26173440 -> 26177536 00010000:00010000:3.0:1789760800.139728:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.139742:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.139751:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.139758:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.139762:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.139772:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.139789:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.139796:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26173440->26177535] (req 26173440->26177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.139873:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca1ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508506240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12805us (12960us total) trans 4294973710 rc 0/0 00000100:00100000:3.0:1789760800.139884:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6422 00000100:00100000:3.0:1789760800.139894:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779210731245 time 779231759188 state 0x2 ref 0 00000100:00100000:2.0:1789760800.141955:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760800.141996:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508506368 00000100:00100000:3.0:1789760800.142005:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508506368 00000100:00100000:3.0:1789760800.142061:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779210731245 time 779231759188 state 0x0 ref 1 00000100:00100000:3.0:1789760800.142077:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779232019604 time 779231759188 state 0x1 ref 1 00000100:00100000:3.0:1789760800.142101:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779232019604 time 779246811133 state 0x0 ref 1 00000100:00100000:3.0:1789760800.142113:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508506368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760800.142161:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26177536, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760800.142169:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.142184:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760800.142188:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760800.142195:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.142217:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760800.154818:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26177536 -> 26181632 00000001:00080000:1.0:1789760800.154819:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973711 is committed 00010000:00010000:3.0:1789760800.154836:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.154850:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.154860:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.154867:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.154872:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.154882:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.154902:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.154911:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26177536->26181631] (req 26177536->26181631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.155012:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508506368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12896us (13057us total) trans 4294973711 rc 0/0 00000100:00100000:3.0:1789760800.155022:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6423 00000100:00100000:3.0:1789760800.155032:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779232019604 time 779246811133 state 0x2 ref 0 00000100:00100000:0.0:1789760800.158922:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760800.159019:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508506496 00000100:00100000:3.0:1789760800.159029:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508506496 00000100:00100000:3.0:1789760800.159078:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779232019604 time 779246811133 state 0x0 ref 1 00000100:00100000:3.0:1789760800.159093:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779247071549 time 779246811133 state 0x1 ref 1 00000100:00100000:3.0:1789760800.159158:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779247071549 time 779263867734 state 0x0 ref 1 00000100:00100000:3.0:1789760800.159170:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508506496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760800.159215:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26181632, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760800.159224:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.159238:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760800.159243:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760800.159250:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.159269:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760800.175616:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973712 is committed 00002000:00010000:1.0:1789760800.175625:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26181632 -> 26185728 00010000:00010000:1.0:1789760800.175654:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.175668:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.175677:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.175684:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.175689:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.175702:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.175722:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.175729:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001a1692fe/0xb49177c0efe575d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26181632->26185727] (req 26181632->26185727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.175835:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508506496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16661us (16916us total) trans 4294973712 rc 0/0 00000100:00100000:1.0:1789760800.175845:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6424 00000100:00100000:1.0:1789760800.175854:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779247071549 time 779263867734 state 0x2 ref 0 00000100:00100000:0.0:1789760800.178090:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760800.178185:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508506624 00000100:00100000:3.0:1789760800.178201:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508506624 00000100:00100000:3.0:1789760800.178272:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779247071549 time 779263867734 state 0x0 ref 1 00000100:00100000:3.0:1789760800.178327:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779264128150 time 779263867734 state 0x1 ref 1 00000100:00100000:3.0:1789760800.178365:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779264128150 time 779283074618 state 0x0 ref 1 00000100:00100000:3.0:1789760800.178380:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508506624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760800.178439:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26185728, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760800.178450:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.178468:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760800.178473:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760800.178481:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.178498:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760800.195774:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973713 is committed 00002000:00010000:1.0:1789760800.195832:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26185728 -> 26189824 00010000:00010000:1.0:1789760800.195856:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.195872:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.195882:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.195891:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.195898:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.195908:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.195934:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.195943:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b024998d/0xb49177c0efe575de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26185728->26189823] (req 26185728->26189823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.196059:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508506624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 17676us (17968us total) trans 4294973713 rc 0/0 00000100:00100000:1.0:1789760800.196074:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6425 00000100:00100000:1.0:1789760800.196096:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779264128150 time 779283074618 state 0x2 ref 0 00000100:00100000:2.0:1789760800.198104:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760800.198156:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508506752 00000100:00100000:3.0:1789760800.198168:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508506752 00000100:00100000:3.0:1789760800.198227:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779264128150 time 779283074618 state 0x0 ref 1 00000100:00100000:3.0:1789760800.198244:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779283335034 time 779283074618 state 0x1 ref 1 00000100:00100000:3.0:1789760800.198279:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779283335034 time 779302989292 state 0x0 ref 1 00000100:00100000:3.0:1789760800.198321:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508506752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760800.198380:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26189824, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:3.0:1789760800.198390:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe575e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.198418:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760800.198423:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760800.198430:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe575e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.198447:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe575e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.214596:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973714 is committed 00002000:00010000:1.0:1789760800.214694:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26189824 -> 26193920 00010000:00010000:1.0:1789760800.214713:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe575e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.214729:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe575e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.214740:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe575e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.214754:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.214760:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe575e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.214771:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe575e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.214796:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe575e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.214805:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010bd49b3/0xb49177c0efe575e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26189824->26193919] (req 26189824->26193919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.214932:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508506752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16606us (16830us total) trans 4294973714 rc 0/0 00000100:00100000:1.0:1789760800.214945:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6426 00000100:00100000:1.0:1789760800.214957:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779283335034 time 779302989292 state 0x2 ref 0 00000100:00100000:0.0:1789760800.216979:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.217058:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508506880 00000100:00100000:1.0:1789760800.217082:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508506880 00000100:00100000:1.0:1789760800.217163:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779283335034 time 779302989292 state 0x0 ref 1 00000100:00100000:1.0:1789760800.217182:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779303249708 time 779302989292 state 0x1 ref 1 00000100:00100000:1.0:1789760800.217221:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779303249708 time 779321930785 state 0x0 ref 1 00000100:00100000:1.0:1789760800.217240:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508506880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.217380:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26193920, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760800.217389:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.217409:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.217414:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.217422:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.217438:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760800.232552:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973715 is committed 00002000:00010000:1.0:1789760800.232602:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26193920 -> 26198016 00010000:00010000:1.0:1789760800.232621:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.232704:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.232716:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.232723:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.232727:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.232737:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.232756:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.232764:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26193920->26198015] (req 26193920->26198015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.232863:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508506880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15623us (15889us total) trans 4294973715 rc 0/0 00000100:00100000:1.0:1789760800.232873:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6427 00000100:00100000:1.0:1789760800.232882:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779303249708 time 779321930785 state 0x2 ref 0 00000100:00100000:0.0:1789760800.236276:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.236355:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508507008 00000100:00100000:1.0:1789760800.236365:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508507008 00000100:00100000:1.0:1789760800.236418:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779303249708 time 779321930785 state 0x0 ref 1 00000100:00100000:1.0:1789760800.236434:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779322191201 time 779321930785 state 0x1 ref 1 00000100:00100000:1.0:1789760800.236461:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779322191201 time 779341171365 state 0x0 ref 1 00000100:00100000:1.0:1789760800.236472:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508507008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.236517:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26198016, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760800.236526:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe575f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.236542:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.236547:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.236553:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe575f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.236567:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe575f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760800.251093:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973716 is committed 00002000:00010000:1.0:1789760800.251183:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26198016 -> 26202112 00010000:00010000:1.0:1789760800.251202:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe575f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.251217:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe575f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.251231:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe575f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.251239:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.251245:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe575f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.251256:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe575f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.251281:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe575f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.251324:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe575f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26198016->26202111] (req 26198016->26202111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.251443:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508507008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14965us (15167us total) trans 4294973716 rc 0/0 00000100:00100000:1.0:1789760800.251455:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6428 00000100:00100000:1.0:1789760800.251468:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779322191201 time 779341171365 state 0x2 ref 0 00000100:00100000:2.0:1789760800.253136:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.253205:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508507136 00000100:00100000:1.0:1789760800.253220:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508507136 00000100:00100000:1.0:1789760800.253319:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779322191201 time 779341171365 state 0x0 ref 1 00000100:00100000:1.0:1789760800.253338:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779341431781 time 779341171365 state 0x1 ref 1 00000100:00100000:1.0:1789760800.253373:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779341431781 time 779358074637 state 0x0 ref 1 00000100:00100000:1.0:1789760800.253391:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508507136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.253446:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26202112, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc807 00010000:00010000:1.0:1789760800.253453:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.253473:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.253479:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.253487:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.253509:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760800.266646:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973717 is committed 00002000:00010000:1.0:1789760800.266790:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26202112 -> 26206208 00002000:00010000:1.0:1789760800.266796:0:8108:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 51207 -> 51209 00010000:00010000:1.0:1789760800.266814:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.266830:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.266839:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.266848:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.266853:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.266877:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.266903:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.266912:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe575fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26202112->26206207] (req 26202112->26206207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.267034:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508507136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13641us (13900us total) trans 4294973717 rc 0/0 00000100:00100000:1.0:1789760800.267046:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6429 00000100:00100000:1.0:1789760800.267058:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779341431781 time 779358074637 state 0x2 ref 0 00000100:00100000:0.0:1789760800.269003:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.269065:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508507264 00000100:00100000:1.0:1789760800.269080:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508507264 00000100:00100000:1.0:1789760800.269142:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779341431781 time 779358074637 state 0x0 ref 1 00000100:00100000:1.0:1789760800.269160:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779358335053 time 779358074637 state 0x1 ref 1 00000100:00100000:1.0:1789760800.269192:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779358335053 time 779373900947 state 0x0 ref 1 00000100:00100000:1.0:1789760800.269207:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508507264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.269265:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26206208, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760800.269274:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57601 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.269325:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.269332:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.269390:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57601 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.269408:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57601 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760800.288381:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26206208 -> 26210304 00010000:00010000:1.0:1789760800.288400:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57601 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.288412:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57601 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.288421:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57601 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.288428:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.288432:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57601 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.288441:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57601 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.288459:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57601 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.288466:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57601 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26206208->26210303] (req 26206208->26210303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.288568:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2a1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2408:x1876699508507264:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19347us (19556us total) trans 4294973718 rc 0/0 00000100:00100000:1.0:1789760800.288582:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6430 00000100:00100000:1.0:1789760800.288592:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779358335053 time 779373900947 state 0x2 ref 0 00000001:00080000:0.0:1789760800.288673:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973718 is committed 00000100:00100000:0.0:1789760800.291975:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.292056:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508507392 00000100:00100000:1.0:1789760800.292069:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508507392 00000100:00100000:1.0:1789760800.292132:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779358335053 time 779373900947 state 0x0 ref 1 00000100:00100000:1.0:1789760800.292162:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779374161363 time 779373900947 state 0x1 ref 1 00000100:00100000:1.0:1789760800.292193:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779374161363 time 779396903944 state 0x0 ref 1 00000100:00100000:1.0:1789760800.292206:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508507392:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.292265:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26210304, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc809 00010000:00010000:1.0:1789760800.292275:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57608 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.292315:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.292321:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.292328:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57608 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.292343:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57608 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.309305:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973719 is committed 00002000:00010000:1.0:1789760800.309340:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26210304 -> 26214400 00010000:00010000:1.0:1789760800.309355:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57608 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.309367:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57608 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.309376:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57608 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.309383:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.309388:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57608 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.309396:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57608 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.309414:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57608 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.309422:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57608 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26210304->26214399] (req 26210304->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.309527:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508507392:12345-192.168.203.23@tcp:4:dd.0 Request processed in 17314us (17557us total) trans 4294973719 rc 0/0 00000100:00100000:1.0:1789760800.309537:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6431 00000100:00100000:1.0:1789760800.309548:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779374161363 time 779396903944 state 0x2 ref 0 00000100:00100000:3.0:1789760800.312388:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.312455:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508507520 00000100:00100000:1.0:1789760800.312466:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508507520 00000100:00100000:1.0:1789760800.312515:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779374161363 time 779396903944 state 0x0 ref 1 00000100:00100000:1.0:1789760800.312530:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779397164360 time 779396903944 state 0x1 ref 1 00000100:00100000:1.0:1789760800.312562:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779397164360 time 779417272287 state 0x0 ref 1 00000100:00100000:1.0:1789760800.312575:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508507520:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.312628:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26214400, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xc807 00010000:00010000:1.0:1789760800.312648:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5760f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.312665:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.312670:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.312684:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5760f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.312707:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5760f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.329093:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973720 is committed 00002000:00010000:1.0:1789760800.329102:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26214400 -> 26218496 00002000:00010000:1.0:1789760800.329108:0:8108:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb blocks from disk: 51207 -> 53257 00010000:00010000:1.0:1789760800.329120:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5760f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.329143:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5760f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.329159:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5760f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.329166:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.329171:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5760f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.329179:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5760f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.329198:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5760f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.329206:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5760f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26214400->26218495] (req 26214400->26218495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.329318:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508507520:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16708us (16901us total) trans 4294973720 rc 0/0 00000100:00100000:1.0:1789760800.329329:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6432 00000100:00100000:1.0:1789760800.329338:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779397164360 time 779417272287 state 0x2 ref 0 00000100:00100000:0.0:1789760800.331179:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.331227:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508507648 00000100:00100000:1.0:1789760800.331237:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508507648 00000100:00100000:1.0:1789760800.331312:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779397164360 time 779417272287 state 0x0 ref 1 00000100:00100000:1.0:1789760800.331328:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779417532703 time 779417272287 state 0x1 ref 1 00000100:00100000:1.0:1789760800.331364:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779417532703 time 779436073375 state 0x0 ref 1 00000100:00100000:1.0:1789760800.331376:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508507648:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.331417:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26218496, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xd009 00010000:00010000:1.0:1789760800.331425:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57616 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.331440:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.331444:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.331451:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57616 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.331464:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57616 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760800.342975:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26218496 -> 26222592 00010000:00010000:1.0:1789760800.342992:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57616 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.343010:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57616 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.343016:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57616 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.343021:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.343026:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57616 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.343032:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57616 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.343045:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57616 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.343050:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57616 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26218496->26222591] (req 26218496->26222591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.343066:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973721 is committed 00000100:00100000:1.0:1789760800.343136:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508507648:12345-192.168.203.23@tcp:4:dd.0 Request processed in 11757us (11955us total) trans 4294973721 rc 0/0 00000100:00100000:1.0:1789760800.343146:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6433 00000100:00100000:1.0:1789760800.343156:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779417532703 time 779436073375 state 0x2 ref 0 00000100:00100000:0.0:1789760800.345252:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.345306:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508507776 00000100:00100000:1.0:1789760800.345313:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508507776 00000100:00100000:1.0:1789760800.345439:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779417532703 time 779436073375 state 0x0 ref 1 00000100:00100000:1.0:1789760800.345451:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779436333791 time 779436073375 state 0x1 ref 1 00000100:00100000:1.0:1789760800.345471:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779436333791 time 779450183221 state 0x0 ref 1 00000100:00100000:1.0:1789760800.345480:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508507776:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.345517:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26222592, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xd009 00010000:00010000:1.0:1789760800.345524:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5761d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.345535:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.345538:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.345542:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5761d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.345551:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5761d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760800.360952:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973722 is committed 00002000:00010000:1.0:1789760800.361048:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26222592 -> 26226688 00010000:00010000:1.0:1789760800.361060:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5761d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.361070:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5761d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.361076:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5761d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.361080:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.361083:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5761d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.361089:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5761d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.361107:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5761d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.361114:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5761d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26222592->26226687] (req 26222592->26226687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.361201:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c9c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508507776:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15717us (15949us total) trans 4294973722 rc 0/0 00000100:00100000:1.0:1789760800.361211:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6434 00000100:00100000:1.0:1789760800.361219:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779436333791 time 779450183221 state 0x2 ref 0 00000100:00100000:3.0:1789760800.364590:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.364649:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508507904 00000100:00100000:1.0:1789760800.364660:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508507904 00000100:00100000:1.0:1789760800.364720:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779436333791 time 779450183221 state 0x0 ref 1 00000100:00100000:1.0:1789760800.364734:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779450443637 time 779450183221 state 0x1 ref 1 00000100:00100000:1.0:1789760800.364763:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779450443637 time 779469473907 state 0x0 ref 1 00000100:00100000:1.0:1789760800.364775:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508507904:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.364825:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26226688, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xd009 00010000:00010000:1.0:1789760800.364834:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57624 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.364848:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.364852:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.364858:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57624 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.364871:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57624 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760800.379574:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973723 is committed 00002000:00010000:2.0:1789760800.379593:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26226688 -> 26230784 00010000:00010000:2.0:1789760800.379609:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57624 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.379627:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57624 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.379648:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57624 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.379656:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.379660:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57624 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.379670:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57624 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.379690:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57624 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.379698:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe57624 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26226688->26230783] (req 26226688->26230783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.379788:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508507904:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15010us (15211us total) trans 4294973723 rc 0/0 00000100:00100000:2.0:1789760800.379798:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6435 00000100:00100000:2.0:1789760800.379808:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779450443637 time 779469473907 state 0x2 ref 0 00000100:00100000:0.0:1789760800.390352:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.390399:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508508032 00000100:00100000:1.0:1789760800.390410:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508508032 00000100:00100000:1.0:1789760800.390520:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779450443637 time 779469473907 state 0x0 ref 1 00000100:00100000:1.0:1789760800.390536:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779469734323 time 779469473907 state 0x1 ref 1 00000100:00100000:1.0:1789760800.390569:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779469734323 time 779495279280 state 0x0 ref 1 00000100:00100000:1.0:1789760800.390583:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508508032:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.390644:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26230784, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xd009 00010000:00010000:1.0:1789760800.390653:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5762b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.390669:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.390674:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.390681:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5762b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.390695:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5762b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760800.413140:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973724 is committed 00002000:00010000:1.0:1789760800.413175:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26230784 -> 26234880 00010000:00010000:1.0:1789760800.413190:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5762b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.413203:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5762b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.413212:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5762b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.413219:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.413225:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5762b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.413242:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5762b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.413262:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5762b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.413271:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe5762b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26230784->26234879] (req 26230784->26234879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.413394:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508508032:12345-192.168.203.23@tcp:4:dd.0 Request processed in 22810us (23047us total) trans 4294973724 rc 0/0 00000100:00100000:1.0:1789760800.413409:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6436 00000100:00100000:1.0:1789760800.413419:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779469734323 time 779495279280 state 0x2 ref 0 00000100:00100000:0.0:1789760800.416382:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.417647:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508508160 00000100:00100000:1.0:1789760800.417661:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508508160 00000100:00100000:1.0:1789760800.417724:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779469734323 time 779495279280 state 0x0 ref 1 00000100:00100000:1.0:1789760800.417738:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779495539696 time 779495279280 state 0x1 ref 1 00000100:00100000:1.0:1789760800.417768:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779495539696 time 779522478466 state 0x0 ref 1 00000100:00100000:1.0:1789760800.417780:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508508160:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.417826:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26234880, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xd009 00010000:00010000:1.0:1789760800.417835:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57632 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.417849:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.417854:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.417860:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57632 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.417872:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57632 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760800.433740:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973725 is committed 00002000:00010000:1.0:1789760800.433875:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26234880 -> 26238976 00010000:00010000:1.0:1789760800.433888:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57632 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.433898:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57632 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.433905:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57632 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.433909:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.433912:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57632 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.433918:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57632 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.433933:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57632 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.433937:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57632 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26234880->26238975] (req 26234880->26238975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.434010:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508508160:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16230us (17636us total) trans 4294973725 rc 0/0 00000100:00100000:1.0:1789760800.434017:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6437 00000100:00100000:1.0:1789760800.434024:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779495539696 time 779522478466 state 0x2 ref 0 00000100:00100000:3.0:1789760800.437823:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.437915:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508508288 00000100:00100000:2.0:1789760800.437926:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508508288 00000100:00100000:2.0:1789760800.437978:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779495539696 time 779522478466 state 0x0 ref 1 00000100:00100000:2.0:1789760800.437994:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779522738882 time 779522478466 state 0x1 ref 1 00000100:00100000:2.0:1789760800.438024:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779522738882 time 779542735145 state 0x0 ref 1 00000100:00100000:2.0:1789760800.438036:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508508288:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.438084:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26238976, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xd009 00010000:00010000:2.0:1789760800.438094:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57639 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.438108:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.438112:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.438118:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57639 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.438132:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57639 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.454851:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973726 is committed 00002000:00010000:2.0:1789760800.454893:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26238976 -> 26243072 00010000:00010000:2.0:1789760800.454909:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57639 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.454923:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57639 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.454938:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57639 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.454945:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.454950:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57639 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.454959:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57639 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.454979:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57639 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.454988:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57639 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26238976->26243071] (req 26238976->26243071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.455075:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508508288:12345-192.168.203.23@tcp:4:dd.0 Request processed in 17036us (17258us total) trans 4294973726 rc 0/0 00000100:00100000:2.0:1789760800.455086:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6438 00000100:00100000:2.0:1789760800.455096:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779522738882 time 779542735145 state 0x2 ref 0 00000100:00100000:0.0:1789760800.459429:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.459500:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508508416 00000100:00100000:1.0:1789760800.459511:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508508416 00000100:00100000:1.0:1789760800.459568:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779522738882 time 779542735145 state 0x0 ref 1 00000100:00100000:1.0:1789760800.459582:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779542995561 time 779542735145 state 0x1 ref 1 00000100:00100000:1.0:1789760800.459609:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779542995561 time 779564319215 state 0x0 ref 1 00000100:00100000:1.0:1789760800.459621:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508508416:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.459677:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26243072, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xd009 00010000:00010000:1.0:1789760800.459686:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57640 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.459700:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.459704:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.459711:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57640 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.459724:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57640 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760800.475252:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973727 is committed 00002000:00010000:2.0:1789760800.475319:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26243072 -> 26247168 00010000:00010000:2.0:1789760800.475334:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57640 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.475352:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57640 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.475361:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57640 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.475369:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.475374:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57640 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.475383:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57640 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.475402:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57640 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.475410:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe57640 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26243072->26247167] (req 26243072->26247167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.475500:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508508416:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15876us (16076us total) trans 4294973727 rc 0/0 00000100:00100000:2.0:1789760800.475510:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6439 00000100:00100000:2.0:1789760800.475520:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779542995561 time 779564319215 state 0x2 ref 0 00000100:00100000:0.0:1789760800.477369:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.477430:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508508544 00000100:00100000:1.0:1789760800.477440:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508508544 00000100:00100000:1.0:1789760800.477486:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779542995561 time 779564319215 state 0x0 ref 1 00000100:00100000:1.0:1789760800.477500:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779564579631 time 779564319215 state 0x1 ref 1 00000100:00100000:1.0:1789760800.477524:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779564579631 time 779582234960 state 0x0 ref 1 00000100:00100000:1.0:1789760800.477538:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508508544:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.477578:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26247168, mtime: 0x6aad9520, atime: 0x0, ctime: 0x6aad9520, blocks: 0xd009 00010000:00010000:1.0:1789760800.477586:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57647 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.477600:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.477604:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.477611:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57647 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.477624:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57647 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760800.489586:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973728 is committed 00002000:00010000:2.0:1789760800.489623:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26247168 -> 26251264 00002000:00010000:2.0:1789760800.489644:0:14639:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760800 -> 1789760801 00002000:00010000:2.0:1789760800.489647:0:14639:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760800 -> 1789760801 00010000:00010000:2.0:1789760800.489660:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57647 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.489673:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57647 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.489681:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57647 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.489697:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.489703:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57647 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.489711:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57647 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.489729:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57647 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.489737:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efb6d352/0xb49177c0efe57647 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26247168->26251263] (req 26247168->26251263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.489824:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b645880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508508544:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12285us (12457us total) trans 4294973728 rc 0/0 00000100:00100000:2.0:1789760800.489835:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6440 00000100:00100000:2.0:1789760800.489845:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779564579631 time 779582234960 state 0x2 ref 0 00000100:00100000:3.0:1789760800.491543:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.491591:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508508672 00000100:00100000:2.0:1789760800.491601:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508508672 00000100:00100000:2.0:1789760800.491655:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779564579631 time 779582234960 state 0x0 ref 1 00000100:00100000:2.0:1789760800.491666:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779582495376 time 779582234960 state 0x1 ref 1 00000100:00100000:2.0:1789760800.491699:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779582495376 time 779596409336 state 0x0 ref 1 00000100:00100000:2.0:1789760800.491711:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508508672:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.491748:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26251264, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.491756:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5764e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.491768:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.491773:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.491779:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5764e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.491792:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5764e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760800.503189:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973729 is committed 00002000:00010000:2.0:1789760800.503258:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26251264 -> 26255360 00010000:00010000:2.0:1789760800.503274:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5764e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.503330:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5764e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.503339:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5764e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.503346:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.503351:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5764e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.503360:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5764e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.503379:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5764e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.503387:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5764e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26251264->26255359] (req 26251264->26255359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.503468:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508508672:12345-192.168.203.23@tcp:4:dd.0 Request processed in 11754us (11926us total) trans 4294973729 rc 0/0 00000100:00100000:2.0:1789760800.503480:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6441 00000100:00100000:2.0:1789760800.503489:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779582495376 time 779596409336 state 0x2 ref 0 00000100:00100000:0.0:1789760800.505226:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.505271:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508508800 00000100:00100000:2.0:1789760800.505279:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508508800 00000100:00100000:2.0:1789760800.505331:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779582495376 time 779596409336 state 0x0 ref 1 00000100:00100000:2.0:1789760800.505341:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779596669752 time 779596409336 state 0x1 ref 1 00000100:00100000:2.0:1789760800.505362:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779596669752 time 779610073595 state 0x0 ref 1 00000100:00100000:2.0:1789760800.505374:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508508800:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.505413:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26255360, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.505421:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57655 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.505430:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.505434:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.505448:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57655 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.505465:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57655 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.510425:0:6455:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760800.515145:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973730 is committed 00002000:00010000:2.0:1789760800.515157:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26255360 -> 26259456 00010000:00010000:2.0:1789760800.515172:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57655 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.515181:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57655 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.515196:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57655 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.515203:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.515208:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57655 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.515216:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57655 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.515235:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57655 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.515242:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57655 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26255360->26259455] (req 26255360->26259455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.515361:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b646a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508508800:12345-192.168.203.23@tcp:4:dd.0 Request processed in 9984us (10132us total) trans 4294973730 rc 0/0 00000100:00100000:2.0:1789760800.515371:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6442 00000100:00100000:2.0:1789760800.515381:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779596669752 time 779610073595 state 0x2 ref 0 00000100:00100000:0.0:1789760800.517534:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.518473:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508508928 00000100:00100000:2.0:1789760800.518483:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508508928 00000100:00100000:2.0:1789760800.518532:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779596669752 time 779610073595 state 0x0 ref 1 00000100:00100000:2.0:1789760800.518543:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779610334011 time 779610073595 state 0x1 ref 1 00000100:00100000:2.0:1789760800.518574:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779610334011 time 779623285030 state 0x0 ref 1 00000100:00100000:2.0:1789760800.518587:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508508928:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.518643:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26259456, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.518651:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5765c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.518663:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.518666:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.518670:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5765c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.518681:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5765c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760800.531708:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26259456 -> 26263552 00000001:00080000:0.0:1789760800.531722:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973731 is committed 00010000:00010000:2.0:1789760800.531728:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5765c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.531742:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5765c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.531751:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5765c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.531759:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.531764:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5765c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.531771:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5765c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.531786:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5765c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.531790:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5765c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26259456->26263551] (req 26259456->26263551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.531864:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b647100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508508928:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13276us (14329us total) trans 4294973731 rc 0/0 00000100:00100000:2.0:1789760800.531874:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6443 00000100:00100000:2.0:1789760800.531888:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779610334011 time 779623285030 state 0x2 ref 0 00000100:00100000:0.0:1789760800.533812:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.533853:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508509056 00000100:00100000:2.0:1789760800.533861:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508509056 00000100:00100000:2.0:1789760800.533900:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779610334011 time 779623285030 state 0x0 ref 1 00000100:00100000:2.0:1789760800.533911:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779623545446 time 779623285030 state 0x1 ref 1 00000100:00100000:2.0:1789760800.533929:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779623545446 time 779638640182 state 0x0 ref 1 00000100:00100000:2.0:1789760800.533938:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508509056:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.533973:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26263552, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.533980:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57663 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.534003:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.534007:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.534013:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57663 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.534027:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57663 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.543883:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973732 is committed 00002000:00010000:2.0:1789760800.543914:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26263552 -> 26267648 00010000:00010000:2.0:1789760800.543929:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57663 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.543943:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57663 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.543951:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57663 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.543958:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.543963:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57663 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.543972:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57663 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.543996:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57663 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.544004:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57663 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26263552->26267647] (req 26263552->26267647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.544085:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b644000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508509056:12345-192.168.203.23@tcp:4:dd.0 Request processed in 10143us (10274us total) trans 4294973732 rc 0/0 00000100:00100000:2.0:1789760800.544095:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6444 00000100:00100000:2.0:1789760800.544104:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779623545446 time 779638640182 state 0x2 ref 0 00000100:00100000:0.0:1789760800.545929:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.546078:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508509184 00000100:00100000:2.0:1789760800.546087:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508509184 00000100:00100000:2.0:1789760800.546134:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779623545446 time 779638640182 state 0x0 ref 1 00000100:00100000:2.0:1789760800.546148:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779638900598 time 779638640182 state 0x1 ref 1 00000100:00100000:2.0:1789760800.546174:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779638900598 time 779650884840 state 0x0 ref 1 00000100:00100000:2.0:1789760800.546184:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508509184:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.546220:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26267648, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.546226:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5766a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.546239:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.546244:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.546250:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5766a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.546263:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5766a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.558547:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973733 is committed 00002000:00010000:2.0:1789760800.558568:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26267648 -> 26271744 00010000:00010000:2.0:1789760800.558581:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5766a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.558589:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5766a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.558595:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5766a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.558599:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.558602:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5766a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.558608:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5766a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.558622:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5766a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.558627:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe5766a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26267648->26271743] (req 26267648->26271743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.558776:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508509184:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12589us (12851us total) trans 4294973733 rc 0/0 00000100:00100000:2.0:1789760800.558786:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6445 00000100:00100000:2.0:1789760800.558794:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779638900598 time 779650884840 state 0x2 ref 0 00000100:00100000:0.0:1789760800.561165:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.561214:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508509312 00000100:00100000:2.0:1789760800.561224:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508509312 00000100:00100000:2.0:1789760800.561270:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779638900598 time 779650884840 state 0x0 ref 1 00000100:00100000:2.0:1789760800.561282:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779651145256 time 779650884840 state 0x1 ref 1 00000100:00100000:2.0:1789760800.561325:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779651145256 time 779666036001 state 0x0 ref 1 00000100:00100000:2.0:1789760800.561337:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508509312:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.561380:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26271744, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.561389:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57671 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.561403:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.561407:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.561413:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57671 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.561424:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57671 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760800.577502:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973734 is committed 00002000:00010000:1.0:1789760800.577570:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26271744 -> 26275840 00010000:00010000:1.0:1789760800.577585:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57671 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.577598:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57671 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.577606:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57671 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.577611:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.577615:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57671 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.577622:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57671 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.577650:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57671 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.577655:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015630f53/0xb49177c0efe57671 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26271744->26275839] (req 26271744->26275839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.577728:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508509312:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16389us (16566us total) trans 4294973734 rc 0/0 00000100:00100000:1.0:1789760800.577744:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6446 00000100:00100000:1.0:1789760800.577753:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779651145256 time 779666036001 state 0x2 ref 0 00000100:00100000:0.0:1789760800.579964:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.580035:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508509440 00000100:00100000:2.0:1789760800.580045:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508509440 00000100:00100000:2.0:1789760800.580102:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779651145256 time 779666036001 state 0x0 ref 1 00000100:00100000:2.0:1789760800.580118:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779666296417 time 779666036001 state 0x1 ref 1 00000100:00100000:2.0:1789760800.580141:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779666296417 time 779684852492 state 0x0 ref 1 00000100:00100000:2.0:1789760800.580154:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508509440:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.580199:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26275840, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.580208:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57678 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.580222:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.580227:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.580234:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57678 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.580249:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57678 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760800.598465:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973735 is committed 00002000:00010000:2.0:1789760800.598511:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26275840 -> 26279936 00010000:00010000:2.0:1789760800.598528:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57678 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.598546:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57678 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.598554:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57678 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.598565:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.598571:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57678 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.598580:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57678 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.598614:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57678 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.598622:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57678 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26275840->26279935] (req 26275840->26279935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.598733:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508509440:12345-192.168.203.23@tcp:4:dd.0 Request processed in 18577us (18775us total) trans 4294973735 rc 0/0 00000100:00100000:2.0:1789760800.598743:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6447 00000100:00100000:2.0:1789760800.598752:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779666296417 time 779684852492 state 0x2 ref 0 00000100:00100000:0.0:1789760800.601549:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.601604:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508509568 00000100:00100000:2.0:1789760800.601615:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508509568 00000100:00100000:2.0:1789760800.601679:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779666296417 time 779684852492 state 0x0 ref 1 00000100:00100000:2.0:1789760800.601695:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779685112908 time 779684852492 state 0x1 ref 1 00000100:00100000:2.0:1789760800.601717:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779685112908 time 779706428515 state 0x0 ref 1 00000100:00100000:2.0:1789760800.601727:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508509568:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.601768:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26279936, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.601774:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5767f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.601787:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.601791:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.601795:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5767f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.601806:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5767f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760800.618301:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26279936 -> 26284032 00010000:00010000:2.0:1789760800.618325:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5767f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.618339:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5767f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.618354:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5767f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.618361:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.618366:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5767f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.618374:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5767f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.618393:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5767f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.618398:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5767f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26279936->26284031] (req 26279936->26284031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.618481:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+7:2407:x1876699508509568:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16749us (17038us total) trans 4294973736 rc 0/0 00000100:00100000:2.0:1789760800.618493:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6448 00000100:00100000:2.0:1789760800.618503:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779685112908 time 779706428515 state 0x2 ref 0 00000001:00080000:3.0:1789760800.620628:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973736 is committed 00000100:00100000:0.0:1789760800.620749:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.621615:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508509696 00000100:00100000:1.0:1789760800.621625:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508509696 00000100:00100000:1.0:1789760800.621692:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779685112908 time 779706428515 state 0x0 ref 1 00000100:00100000:1.0:1789760800.621708:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779706688931 time 779706428515 state 0x1 ref 1 00000100:00100000:1.0:1789760800.621733:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779706688931 time 779726444189 state 0x0 ref 1 00000100:00100000:1.0:1789760800.621747:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508509696:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.621792:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26284032, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760800.621801:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57686 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.621817:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.621821:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.621828:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57686 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.621842:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57686 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760800.637660:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973737 is committed 00002000:00010000:2.0:1789760800.637817:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26284032 -> 26288128 00010000:00010000:2.0:1789760800.637849:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57686 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.637871:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57686 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.637881:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57686 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.637888:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.637894:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57686 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.637905:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57686 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.637931:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57686 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.637939:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e40846ba/0xb49177c0efe57686 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26284032->26288127] (req 26284032->26288127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.638045:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508509696:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16295us (17292us total) trans 4294973737 rc 0/0 00000100:00100000:2.0:1789760800.638057:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6449 00000100:00100000:2.0:1789760800.638077:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779706688931 time 779726444189 state 0x2 ref 0 00000100:00100000:0.0:1789760800.638449:0:3475:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffffa049beaf9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534095872:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760800.638531:0:3475:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760800.638578:0:6472:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699534095872 00000100:00100000:1.0:1789760800.638586:0:6472:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699534095872 00000100:00100000:1.0:1789760800.638644:0:6472:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 477 00000100:00100000:1.0:1789760800.638657:0:6472:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534095872:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760800.638693:0:6472:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261778432 avail 2259681280 00000100:00100000:1.0:1789760800.638741:0:6472:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3475:x1876699534095872:12345-0@lo:41:osp-pre-1-0.0 Request processed in 88us (210us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760800.638749:0:6472:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 477 00000100:00100000:0.0:1789760800.638793:0:3475:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffffa049beaf9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3475:1876699534095872:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760800.639953:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.640001:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508509824 00000100:00100000:2.0:1789760800.640012:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508509824 00000100:00100000:2.0:1789760800.640074:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779706688931 time 779726444189 state 0x0 ref 1 00000100:00100000:2.0:1789760800.640088:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779726704605 time 779726444189 state 0x1 ref 1 00000100:00100000:2.0:1789760800.640111:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779726704605 time 779744823294 state 0x0 ref 1 00000100:00100000:2.0:1789760800.640122:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508509824:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.640167:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26288128, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.640178:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5768d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.640200:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.640205:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.640212:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5768d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.640226:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5768d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.656254:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973738 is committed 00002000:00010000:2.0:1789760800.656329:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26288128 -> 26292224 00010000:00010000:2.0:1789760800.656344:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5768d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.656357:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5768d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.656371:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5768d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.656379:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.656385:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5768d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.656394:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5768d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.656414:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5768d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.656422:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5768d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26288128->26292223] (req 26288128->26292223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.656504:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508509824:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16379us (16549us total) trans 4294973738 rc 0/0 00000100:00100000:2.0:1789760800.656515:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6450 00000100:00100000:2.0:1789760800.656525:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779726704605 time 779744823294 state 0x2 ref 0 00000100:00100000:0.0:1789760800.661217:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.661272:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508509952 00000100:00100000:2.0:1789760800.661282:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508509952 00000100:00100000:2.0:1789760800.661367:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779726704605 time 779744823294 state 0x0 ref 1 00000100:00100000:2.0:1789760800.661383:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779745083710 time 779744823294 state 0x1 ref 1 00000100:00100000:2.0:1789760800.661410:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779745083710 time 779766121290 state 0x0 ref 1 00000100:00100000:2.0:1789760800.661422:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508509952:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.661472:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26292224, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.661481:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57694 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.661497:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.661502:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.661509:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57694 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.661523:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57694 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.682004:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973739 is committed 00002000:00010000:2.0:1789760800.682042:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26292224 -> 26296320 00010000:00010000:2.0:1789760800.682057:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57694 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.682070:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57694 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.682079:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57694 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.682087:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.682091:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57694 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.682099:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57694 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.682118:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57694 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.682127:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe57694 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26292224->26296319] (req 26292224->26296319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.682206:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891ff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508509952:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20781us (20993us total) trans 4294973739 rc 0/0 00000100:00100000:2.0:1789760800.682217:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6451 00000100:00100000:2.0:1789760800.682227:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779745083710 time 779766121290 state 0x2 ref 0 00000100:00100000:0.0:1789760800.684564:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.684621:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508510080 00000100:00100000:2.0:1789760800.684643:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508510080 00000100:00100000:2.0:1789760800.684700:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779745083710 time 779766121290 state 0x0 ref 1 00000100:00100000:2.0:1789760800.684716:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779766381706 time 779766121290 state 0x1 ref 1 00000100:00100000:2.0:1789760800.684741:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779766381706 time 779789452352 state 0x0 ref 1 00000100:00100000:2.0:1789760800.684750:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508510080:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.684786:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26296320, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.684793:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5769b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.684804:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.684807:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.684811:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5769b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.684821:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5769b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.702321:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973740 is committed 00002000:00010000:2.0:1789760800.702558:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26296320 -> 26300416 00010000:00010000:2.0:1789760800.702577:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5769b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.702591:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5769b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.702600:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5769b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.702607:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.702612:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5769b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.702621:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5769b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.702659:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5769b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.702667:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe5769b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26296320->26300415] (req 26296320->26300415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.702877:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508510080:12345-192.168.203.23@tcp:4:dd.0 Request processed in 18121us (18315us total) trans 4294973740 rc 0/0 00000100:00100000:2.0:1789760800.702890:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6452 00000100:00100000:2.0:1789760800.702899:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779766381706 time 779789452352 state 0x2 ref 0 00000100:00100000:3.0:1789760800.708089:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.708160:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508510208 00000100:00100000:2.0:1789760800.708170:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508510208 00000100:00100000:2.0:1789760800.708232:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779766381706 time 779789452352 state 0x0 ref 1 00000100:00100000:2.0:1789760800.708247:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779789712768 time 779789452352 state 0x1 ref 1 00000100:00100000:2.0:1789760800.708276:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779789712768 time 779812986480 state 0x0 ref 1 00000100:00100000:2.0:1789760800.708308:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508510208:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.708372:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26300416, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.708381:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.708396:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.708401:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.708408:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.708420:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.723943:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973741 is committed 00002000:00010000:2.0:1789760800.723969:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26300416 -> 26304512 00010000:00010000:2.0:1789760800.723984:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.723997:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.724006:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.724013:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.724018:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.724028:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.724045:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.724052:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26300416->26304511] (req 26300416->26304511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.724133:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508510208:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15844us (16051us total) trans 4294973741 rc 0/0 00000100:00100000:2.0:1789760800.724143:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6453 00000100:00100000:2.0:1789760800.724153:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779789712768 time 779812986480 state 0x2 ref 0 00000100:00100000:0.0:1789760800.726093:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.726132:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508510336 00000100:00100000:2.0:1789760800.726141:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508510336 00000100:00100000:2.0:1789760800.726187:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779789712768 time 779812986480 state 0x0 ref 1 00000100:00100000:2.0:1789760800.726198:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779813246896 time 779812986480 state 0x1 ref 1 00000100:00100000:2.0:1789760800.726219:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779813246896 time 779830929907 state 0x0 ref 1 00000100:00100000:2.0:1789760800.726239:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508510336:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.726278:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26304512, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.726309:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe576a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.726322:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.726327:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.726333:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe576a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.726345:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe576a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.741388:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973742 is committed 00002000:00010000:3.0:1789760800.741417:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26304512 -> 26308608 00010000:00010000:3.0:1789760800.741433:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe576a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.741445:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe576a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.741453:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe576a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.741459:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.741464:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe576a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.741472:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe576a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.741490:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe576a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.741501:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063ec5736/0xb49177c0efe576a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26304512->26308607] (req 26304512->26308607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.741604:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049891fdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508510336:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15369us (15509us total) trans 4294973742 rc 0/0 00000100:00100000:3.0:1789760800.741615:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6454 00000100:00100000:3.0:1789760800.741647:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779813246896 time 779830929907 state 0x2 ref 0 00000100:00100000:1.0:1789760800.743440:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.743496:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508510464 00000100:00100000:2.0:1789760800.743506:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508510464 00000100:00100000:2.0:1789760800.743554:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779813246896 time 779830929907 state 0x0 ref 1 00000100:00100000:2.0:1789760800.743570:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779831190323 time 779830929907 state 0x1 ref 1 00000100:00100000:2.0:1789760800.744346:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779831190323 time 779849050137 state 0x0 ref 1 00000100:00100000:2.0:1789760800.744363:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508510464:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.744406:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26308608, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.744414:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.744435:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.744448:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.744454:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.744470:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.758048:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973743 is committed 00002000:00010000:2.0:1789760800.758064:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26308608 -> 26312704 00010000:00010000:2.0:1789760800.758080:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.758098:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.758108:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.758115:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.758120:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.758129:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.758149:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.758157:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26308608->26312703] (req 26308608->26312703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.758252:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508510464:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13880us (14802us total) trans 4294973743 rc 0/0 00000100:00100000:2.0:1789760800.758262:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6455 00000100:00100000:2.0:1789760800.758283:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779831190323 time 779849050137 state 0x2 ref 0 00000100:00100000:0.0:1789760800.760442:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.760499:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508510592 00000100:00100000:2.0:1789760800.760510:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508510592 00000100:00100000:2.0:1789760800.760560:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779831190323 time 779849050137 state 0x0 ref 1 00000100:00100000:2.0:1789760800.760574:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779849310553 time 779849050137 state 0x1 ref 1 00000100:00100000:2.0:1789760800.760609:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779849310553 time 779865320049 state 0x0 ref 1 00000100:00100000:2.0:1789760800.760621:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508510592:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.760685:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26312704, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.760695:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe576b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.760709:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.760713:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.760726:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe576b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.760740:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe576b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.780897:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973744 is committed 00002000:00010000:1.0:1789760800.781040:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26312704 -> 26316800 00010000:00010000:1.0:1789760800.781055:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe576b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.781067:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe576b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.781082:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe576b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.781088:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.781093:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe576b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.781101:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe576b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.781120:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe576b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.781127:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cebf4bc/0xb49177c0efe576b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26312704->26316799] (req 26312704->26316799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.781221:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e65f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508510592:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20595us (20779us total) trans 4294973744 rc 0/0 00000100:00100000:1.0:1789760800.781230:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6456 00000100:00100000:1.0:1789760800.781238:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779849310553 time 779865320049 state 0x2 ref 0 00000100:00100000:0.0:1789760800.784743:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760800.784797:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508510720 00000100:00100000:3.0:1789760800.784809:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508510720 00000100:00100000:3.0:1789760800.784904:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779849310553 time 779865320049 state 0x0 ref 1 00000100:00100000:3.0:1789760800.784922:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779865580465 time 779865320049 state 0x1 ref 1 00000100:00100000:3.0:1789760800.784957:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779865580465 time 779889667682 state 0x0 ref 1 00000100:00100000:3.0:1789760800.784971:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508510720:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760800.785025:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26316800, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760800.785035:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe576be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.785051:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:3.0:1789760800.785055:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760800.785062:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe576be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.785082:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe576be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.797777:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973745 is committed 00002000:00010000:2.0:1789760800.797817:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26316800 -> 26320896 00010000:00010000:2.0:1789760800.797832:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe576be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.797845:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe576be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.797857:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe576be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.797864:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760800.797868:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe576be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.797875:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe576be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.797892:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe576be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.797899:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b480a82/0xb49177c0efe576be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26316800->26320895] (req 26316800->26320895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760800.797996:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508510720:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13012us (13246us total) trans 4294973745 rc 0/0 00000100:00100000:2.0:1789760800.798006:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6457 00000100:00100000:2.0:1789760800.798016:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779865580465 time 779889667682 state 0x2 ref 0 00000100:00100000:0.0:1789760800.801200:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.801269:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508510848 00000100:00100000:2.0:1789760800.801279:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508510848 00000100:00100000:2.0:1789760800.801351:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779865580465 time 779889667682 state 0x0 ref 1 00000100:00100000:2.0:1789760800.801366:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779889928098 time 779889667682 state 0x1 ref 1 00000100:00100000:2.0:1789760800.801392:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779889928098 time 779906102649 state 0x0 ref 1 00000100:00100000:2.0:1789760800.801407:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e67480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508510848:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.801466:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26320896, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.801474:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.801492:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.801496:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.801503:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.801516:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.821318:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973746 is committed 00002000:00010000:1.0:1789760800.821493:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26320896 -> 26324992 00010000:00010000:1.0:1789760800.821506:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.821525:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.821533:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.821541:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.821546:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.821554:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.821573:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.821581:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001076da99/0xb49177c0efe576c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26320896->26324991] (req 26320896->26324991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.821678:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e67480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508510848:12345-192.168.203.23@tcp:4:dd.0 Request processed in 20271us (20483us total) trans 4294973746 rc 0/0 00000100:00100000:1.0:1789760800.821688:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6458 00000100:00100000:1.0:1789760800.821696:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779889928098 time 779906102649 state 0x2 ref 0 00000100:00100000:0.0:1789760800.828775:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.828949:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508510976 00000100:00100000:1.0:1789760800.828959:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508510976 00000100:00100000:1.0:1789760800.829013:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779889928098 time 779906102649 state 0x0 ref 1 00000100:00100000:1.0:1789760800.829028:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779906363065 time 779906102649 state 0x1 ref 1 00000100:00100000:1.0:1789760800.829059:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779906363065 time 779933769077 state 0x0 ref 1 00000100:00100000:1.0:1789760800.829072:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e64a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508510976:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.829118:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26324992, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760800.829128:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.829143:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.829147:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.829153:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.829167:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.843698:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973747 is committed 00002000:00010000:1.0:1789760800.843744:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26324992 -> 26329088 00010000:00010000:1.0:1789760800.843758:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.843775:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.843788:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.843796:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.843800:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.843808:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.843827:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.843834:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26324992->26329087] (req 26324992->26329087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.843929:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e64a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508510976:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14855us (15158us total) trans 4294973747 rc 0/0 00000100:00100000:1.0:1789760800.843938:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6459 00000100:00100000:1.0:1789760800.843947:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779906363065 time 779933769077 state 0x2 ref 0 00000100:00100000:0.0:1789760800.847987:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760800.848057:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508511104 00000100:00100000:2.0:1789760800.848069:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508511104 00000100:00100000:2.0:1789760800.848134:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779906363065 time 779933769077 state 0x0 ref 1 00000100:00100000:2.0:1789760800.848149:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779934029493 time 779933769077 state 0x1 ref 1 00000100:00100000:2.0:1789760800.848179:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779934029493 time 779952889433 state 0x0 ref 1 00000100:00100000:2.0:1789760800.848188:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887e66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508511104:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760800.848232:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26329088, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:2.0:1789760800.848240:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe576d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.848253:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:2.0:1789760800.848257:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760800.848261:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe576d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760800.848272:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe576d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760800.862142:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973748 is committed 00002000:00010000:1.0:1789760800.862252:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26329088 -> 26333184 00010000:00010000:1.0:1789760800.862267:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe576d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.862327:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe576d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.862336:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe576d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.862343:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.862348:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe576d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.862362:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe576d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.862384:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe576d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.862393:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004377da4b/0xb49177c0efe576d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26329088->26333183] (req 26329088->26333183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.862479:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887e66680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508511104:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14288us (14498us total) trans 4294973748 rc 0/0 00000100:00100000:1.0:1789760800.862490:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6460 00000100:00100000:1.0:1789760800.862498:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779934029493 time 779952889433 state 0x2 ref 0 00000100:00100000:0.0:1789760800.864734:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.864784:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508511232 00000100:00100000:1.0:1789760800.864794:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508511232 00000100:00100000:1.0:1789760800.864841:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779934029493 time 779952889433 state 0x0 ref 1 00000100:00100000:1.0:1789760800.864856:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779953149849 time 779952889433 state 0x1 ref 1 00000100:00100000:1.0:1789760800.864882:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779953149849 time 779969592332 state 0x0 ref 1 00000100:00100000:1.0:1789760800.864894:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd52a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508511232:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.864939:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26333184, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760800.864947:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.864960:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.864964:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.864970:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.864995:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760800.876655:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973749 is committed 00002000:00010000:1.0:1789760800.876674:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26333184 -> 26337280 00010000:00010000:1.0:1789760800.876690:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.876703:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.876717:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.876725:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760800.876730:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.876739:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.876757:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.876764:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26333184->26337279] (req 26333184->26337279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760800.876844:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd52a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508511232:12345-192.168.203.23@tcp:4:dd.0 Request processed in 11949us (12112us total) trans 4294973749 rc 0/0 00000100:00100000:1.0:1789760800.876856:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6461 00000100:00100000:1.0:1789760800.876866:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779953149849 time 779969592332 state 0x2 ref 0 00000100:00100000:0.0:1789760800.878796:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.878842:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508511360 00000100:00100000:1.0:1789760800.878852:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508511360 00000100:00100000:1.0:1789760800.878897:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779953149849 time 779969592332 state 0x0 ref 1 00000100:00100000:1.0:1789760800.878910:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779969852748 time 779969592332 state 0x1 ref 1 00000100:00100000:1.0:1789760800.878939:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779969852748 time 779983649693 state 0x0 ref 1 00000100:00100000:1.0:1789760800.878951:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508511360:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.878998:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26337280, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760800.879006:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe576e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.879019:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.879023:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.879030:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe576e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.879052:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe576e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760800.889525:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973750 is committed 00002000:00010000:3.0:1789760800.889557:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26337280 -> 26341376 00010000:00010000:3.0:1789760800.889569:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe576e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.889578:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe576e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.889586:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe576e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.889593:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.889598:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe576e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.889607:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe576e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.889694:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe576e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.889703:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000048d5a163/0xb49177c0efe576e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26337280->26341375] (req 26337280->26341375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.889782:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499cd51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508511360:12345-192.168.203.23@tcp:4:dd.0 Request processed in 10830us (10986us total) trans 4294973750 rc 0/0 00000100:00100000:3.0:1789760800.889789:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6462 00000100:00100000:3.0:1789760800.889797:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779969852748 time 779983649693 state 0x2 ref 0 00000100:00100000:0.0:1789760800.892546:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.892717:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508511488 00000100:00100000:1.0:1789760800.892733:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508511488 00000100:00100000:1.0:1789760800.892783:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779969852748 time 779983649693 state 0x0 ref 1 00000100:00100000:1.0:1789760800.892794:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779983910109 time 779983649693 state 0x1 ref 1 00000100:00100000:1.0:1789760800.892823:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779983910109 time 779997534093 state 0x0 ref 1 00000100:00100000:1.0:1789760800.892836:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508511488:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.892885:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26341376, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760800.892895:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.892917:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=779s v=(0 0) 00010000:00010000:1.0:1789760800.892922:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.892928:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.892941:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760800.906057:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973751 is committed 00002000:00010000:3.0:1789760800.906080:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26341376 -> 26345472 00010000:00010000:3.0:1789760800.906095:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.906109:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.906118:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.906126:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.906131:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.906141:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.906164:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.906177:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006498ca9a/0xb49177c0efe576e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26341376->26345471] (req 26341376->26345471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.906263:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508511488:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13425us (13724us total) trans 4294973751 rc 0/0 00000100:00100000:3.0:1789760800.906275:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6463 00000100:00100000:3.0:1789760800.906321:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779983910109 time 779997534093 state 0x2 ref 0 00000100:00100000:0.0:1789760800.909048:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760800.909266:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508511616 00000100:00100000:1.0:1789760800.909275:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508511616 00000100:00100000:1.0:1789760800.909360:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779983910109 time 779997534093 state 0x0 ref 1 00000100:00100000:1.0:1789760800.909375:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779997794509 time 779997534093 state 0x1 ref 1 00000100:00100000:1.0:1789760800.909396:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779997794509 time 780014107145 state 0x0 ref 1 00000100:00100000:1.0:1789760800.909408:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508511616:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760800.909460:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26345472, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760800.909469:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe576ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.909484:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760800.909489:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760800.909495:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe576ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760800.909511:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe576ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.921872:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973752 is committed 00002000:00010000:3.0:1789760800.921960:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26345472 -> 26349568 00010000:00010000:3.0:1789760800.921977:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe576ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.921991:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe576ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.921999:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe576ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.922006:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.922011:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe576ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.922020:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe576ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.922045:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe576ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.922053:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002feaf04a/0xb49177c0efe576ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26345472->26349567] (req 26345472->26349567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.922170:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508511616:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12757us (13123us total) trans 4294973752 rc 0/0 00000100:00100000:3.0:1789760800.922180:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6464 00000100:00100000:3.0:1789760800.922191:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779997794509 time 780014107145 state 0x2 ref 0 00000100:00100000:0.0:1789760800.924605:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760800.924675:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508511744 00000100:00100000:3.0:1789760800.924699:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508511744 00000100:00100000:3.0:1789760800.924749:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 779997794509 time 780014107145 state 0x0 ref 1 00000100:00100000:3.0:1789760800.924761:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780014367561 time 780014107145 state 0x1 ref 1 00000100:00100000:3.0:1789760800.924784:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780014367561 time 780029495615 state 0x0 ref 1 00000100:00100000:3.0:1789760800.924827:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508511744:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760800.924867:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26349568, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760800.924875:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe576f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.924889:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760800.924894:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760800.924900:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe576f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.924914:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe576f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760800.943692:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973753 is committed 00002000:00010000:3.0:1789760800.943783:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26349568 -> 26353664 00010000:00010000:3.0:1789760800.943798:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe576f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.943815:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe576f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.943824:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe576f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.943831:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.943836:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe576f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.943844:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe576f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.943862:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe576f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.943869:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe576f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26349568->26353663] (req 26349568->26353663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.943995:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508511744:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19198us (19394us total) trans 4294973753 rc 0/0 00000100:00100000:3.0:1789760800.944006:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6465 00000100:00100000:3.0:1789760800.944017:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780014367561 time 780029495615 state 0x2 ref 0 00000100:00100000:0.0:1789760800.946601:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760800.946770:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508511872 00000100:00100000:3.0:1789760800.946779:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508511872 00000100:00100000:3.0:1789760800.946832:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780014367561 time 780029495615 state 0x0 ref 1 00000100:00100000:3.0:1789760800.946848:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780029756031 time 780029495615 state 0x1 ref 1 00000100:00100000:3.0:1789760800.946879:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780029756031 time 780051589936 state 0x0 ref 1 00000100:00100000:3.0:1789760800.946892:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508511872:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760800.946939:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26353664, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760800.946948:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe576fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.946963:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760800.946967:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760800.946972:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe576fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.947002:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe576fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.965506:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973754 is committed 00002000:00010000:3.0:1789760800.965573:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26353664 -> 26357760 00010000:00010000:3.0:1789760800.965589:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe576fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.965600:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe576fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.965608:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe576fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.965614:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.965619:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe576fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.965638:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe576fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.965658:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe576fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.965665:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe576fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26353664->26357759] (req 26353664->26357759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.965755:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508511872:12345-192.168.203.23@tcp:4:dd.0 Request processed in 18860us (19157us total) trans 4294973754 rc 0/0 00000100:00100000:3.0:1789760800.965769:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6466 00000100:00100000:3.0:1789760800.965779:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780029756031 time 780051589936 state 0x2 ref 0 00000100:00100000:0.0:1789760800.969241:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760800.969343:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508512000 00000100:00100000:3.0:1789760800.969352:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508512000 00000100:00100000:3.0:1789760800.969403:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780029756031 time 780051589936 state 0x0 ref 1 00000100:00100000:3.0:1789760800.969417:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780051850352 time 780051589936 state 0x1 ref 1 00000100:00100000:3.0:1789760800.969436:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780051850352 time 780074147926 state 0x0 ref 1 00000100:00100000:3.0:1789760800.969482:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508512000:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760800.969519:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26357760, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760800.969526:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57704 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.969539:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760800.969640:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760800.969649:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57704 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.969664:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57704 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760800.985353:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973755 is committed 00002000:00010000:3.0:1789760800.985396:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26357760 -> 26361856 00010000:00010000:3.0:1789760800.985411:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57704 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.985424:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57704 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.985433:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57704 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.985440:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.985444:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57704 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.985454:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57704 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.985472:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57704 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.985480:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57704 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26357760->26361855] (req 26357760->26361855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.985565:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508512000:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16079us (16328us total) trans 4294973755 rc 0/0 00000100:00100000:3.0:1789760800.985577:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6467 00000100:00100000:3.0:1789760800.985597:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780051850352 time 780074147926 state 0x2 ref 0 00000100:00100000:0.0:1789760800.987243:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760800.987367:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508512128 00000100:00100000:3.0:1789760800.987376:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508512128 00000100:00100000:3.0:1789760800.987420:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780051850352 time 780074147926 state 0x0 ref 1 00000100:00100000:3.0:1789760800.987432:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780074408342 time 780074147926 state 0x1 ref 1 00000100:00100000:3.0:1789760800.987452:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780074408342 time 780092163598 state 0x0 ref 1 00000100:00100000:3.0:1789760800.987463:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508512128:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760800.987503:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26361856, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760800.987510:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5770b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.987525:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760800.987529:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760800.987535:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5770b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.987547:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5770b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760800.998210:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973756 is committed 00002000:00010000:3.0:1789760800.998242:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26361856 -> 26365952 00010000:00010000:3.0:1789760800.998257:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5770b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.998269:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5770b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.998282:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5770b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.998315:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760800.998320:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5770b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.998329:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5770b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.998347:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5770b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760800.998355:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe5770b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26361856->26365951] (req 26361856->26365951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760800.998434:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508512128:12345-192.168.203.23@tcp:4:dd.0 Request processed in 10967us (11189us total) trans 4294973756 rc 0/0 00000100:00100000:3.0:1789760800.998445:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6468 00000100:00100000:3.0:1789760800.998454:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780074408342 time 780092163598 state 0x2 ref 0 00000100:00100000:0.0:1789760801.000787:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.000821:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508512256 00000100:00100000:3.0:1789760801.000830:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508512256 00000100:00100000:3.0:1789760801.000881:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780074408342 time 780092163598 state 0x0 ref 1 00000100:00080000:0.0:1789760801.000882:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760801.000895:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780092424014 time 780092163598 state 0x1 ref 1 00000100:00100000:3.0:1789760801.000913:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780092424014 time 780105625368 state 0x0 ref 1 00000100:00100000:3.0:1789760801.000923:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c1fc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508512256:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.000968:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26365952, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760801.000975:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57712 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.000989:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760801.000993:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.000999:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57712 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.001013:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57712 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760801.012419:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973757 is committed 00002000:00010000:3.0:1789760801.012505:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26365952 -> 26370048 00010000:00010000:3.0:1789760801.012520:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57712 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.012531:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57712 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.012540:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57712 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.012547:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.012552:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57712 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.012561:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57712 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.012581:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57712 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.012588:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57712 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26365952->26370047] (req 26365952->26370047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.012693:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c1fc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508512256:12345-192.168.203.23@tcp:4:dd.0 Request processed in 11766us (11908us total) trans 4294973757 rc 0/0 00000100:00100000:3.0:1789760801.012702:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6469 00000100:00100000:3.0:1789760801.012712:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780092424014 time 780105625368 state 0x2 ref 0 00000100:00100000:0.0:1789760801.016314:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.016344:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508512384 00000100:00100000:3.0:1789760801.016354:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508512384 00000100:00100000:3.0:1789760801.016404:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780092424014 time 780105625368 state 0x0 ref 1 00000100:00100000:3.0:1789760801.016418:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780105885784 time 780105625368 state 0x1 ref 1 00000100:00100000:3.0:1789760801.016444:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780105885784 time 780121155176 state 0x0 ref 1 00000100:00100000:3.0:1789760801.016459:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508512384:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.016503:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26370048, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760801.016512:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57719 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.016527:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760801.016532:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.016538:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57719 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.016552:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57719 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.031049:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973758 is committed 00002000:00010000:3.0:1789760801.031087:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26370048 -> 26374144 00010000:00010000:3.0:1789760801.031104:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57719 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.031117:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57719 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.031126:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57719 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.031133:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.031138:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57719 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.031147:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57719 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.031171:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57719 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.031179:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57719 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26370048->26374143] (req 26370048->26374143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.031265:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508512384:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14805us (14981us total) trans 4294973758 rc 0/0 00000100:00100000:3.0:1789760801.031276:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6470 00000100:00100000:3.0:1789760801.031320:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780105885784 time 780121155176 state 0x2 ref 0 00000100:00100000:0.0:1789760801.033278:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.033436:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508512512 00000100:00100000:3.0:1789760801.033446:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508512512 00000100:00100000:3.0:1789760801.033495:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780105885784 time 780121155176 state 0x0 ref 1 00000100:00100000:3.0:1789760801.033514:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780121415592 time 780121155176 state 0x1 ref 1 00000100:00100000:3.0:1789760801.033537:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780121415592 time 780138248495 state 0x0 ref 1 00000100:00100000:3.0:1789760801.033549:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508512512:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.033592:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26374144, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760801.033599:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57720 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.033614:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760801.033618:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.033624:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57720 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.034375:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57720 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.049013:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973759 is committed 00002000:00010000:0.0:1789760801.049109:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26374144 -> 26378240 00010000:00010000:0.0:1789760801.049123:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57720 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.049136:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57720 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.049146:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57720 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.049153:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760801.049158:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57720 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.049167:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57720 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.049188:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57720 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.049196:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bbee2f2/0xb49177c0efe57720 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26374144->26378239] (req 26374144->26378239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760801.049339:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508512512:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15734us (16013us total) trans 4294973759 rc 0/0 00000100:00100000:0.0:1789760801.049355:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6471 00000100:00100000:0.0:1789760801.049365:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780121415592 time 780138248495 state 0x2 ref 0 00000100:00100000:0.0:1789760801.051230:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.051282:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508512640 00000100:00100000:3.0:1789760801.051312:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508512640 00000100:00100000:3.0:1789760801.051360:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780121415592 time 780138248495 state 0x0 ref 1 00000100:00100000:3.0:1789760801.051377:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780138508911 time 780138248495 state 0x1 ref 1 00000100:00100000:3.0:1789760801.051405:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780138508911 time 780156115272 state 0x0 ref 1 00000100:00100000:3.0:1789760801.051418:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508512640:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.051472:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26378240, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760801.051480:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57727 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.051494:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760801.051498:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.051505:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57727 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.051519:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57727 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.062538:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973760 is committed 00002000:00010000:1.0:1789760801.062553:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26378240 -> 26382336 00010000:00010000:1.0:1789760801.062569:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57727 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.062582:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57727 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.062591:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57727 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.062599:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.062604:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57727 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.062613:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57727 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.062651:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57727 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.062660:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000279f441c/0xb49177c0efe57727 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26378240->26382335] (req 26378240->26382335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.062738:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508512640:12345-192.168.203.23@tcp:4:dd.0 Request processed in 11319us (11509us total) trans 4294973760 rc 0/0 00000100:00100000:1.0:1789760801.062749:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6472 00000100:00100000:1.0:1789760801.062759:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780138508911 time 780156115272 state 0x2 ref 0 00000100:00100000:1.0:1789760801.064392:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.064427:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508512768 00000100:00100000:3.0:1789760801.064435:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508512768 00000100:00100000:3.0:1789760801.064499:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780138508911 time 780156115272 state 0x0 ref 1 00000100:00100000:3.0:1789760801.064512:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780156375688 time 780156115272 state 0x1 ref 1 00000100:00100000:3.0:1789760801.064530:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780156375688 time 780169241611 state 0x0 ref 1 00000100:00100000:3.0:1789760801.064539:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508512768:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.064577:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26382336, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760801.064584:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5772e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.064598:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760801.064602:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.064609:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5772e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.064621:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5772e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.076695:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973761 is committed 00002000:00010000:0.0:1789760801.076772:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26382336 -> 26386432 00010000:00010000:0.0:1789760801.076788:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5772e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.076801:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5772e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.076810:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5772e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.076818:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760801.076822:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5772e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.076831:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5772e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.076851:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5772e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.076860:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b0ec3bdc/0xb49177c0efe5772e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26382336->26386431] (req 26382336->26386431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760801.076954:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beafb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508512768:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12411us (12570us total) trans 4294973761 rc 0/0 00000100:00100000:0.0:1789760801.076965:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6473 00000100:00100000:0.0:1789760801.076976:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780156375688 time 780169241611 state 0x2 ref 0 00000100:00100000:3.0:1789760801.078563:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760801.078610:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508512896 00000100:00100000:0.0:1789760801.078618:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508512896 00000100:00100000:0.0:1789760801.078673:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780156375688 time 780169241611 state 0x0 ref 1 00000100:00100000:0.0:1789760801.078688:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780169502027 time 780169241611 state 0x1 ref 1 00000100:00100000:0.0:1789760801.078710:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780169502027 time 780183422032 state 0x0 ref 1 00000100:00100000:0.0:1789760801.078720:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508512896:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760801.078752:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26386432, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:0.0:1789760801.078757:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57735 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.078768:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760801.078771:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760801.078775:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57735 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.078785:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57735 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.092154:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973762 is committed 00002000:00010000:0.0:1789760801.092203:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26386432 -> 26390528 00010000:00010000:0.0:1789760801.092221:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57735 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.092234:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57735 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.092243:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57735 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.092250:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760801.092255:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57735 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.092264:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57735 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.092313:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57735 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.092323:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57735 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26386432->26390527] (req 26386432->26390527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760801.092412:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508512896:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13689us (13848us total) trans 4294973762 rc 0/0 00000100:00100000:0.0:1789760801.092426:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6474 00000100:00100000:0.0:1789760801.092435:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780169502027 time 780183422032 state 0x2 ref 0 00000100:00100000:2.0:1789760801.095169:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.095268:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508513024 00000100:00100000:1.0:1789760801.095278:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508513024 00000100:00100000:1.0:1789760801.095362:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780169502027 time 780183422032 state 0x0 ref 1 00000100:00100000:1.0:1789760801.095377:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780183682448 time 780183422032 state 0x1 ref 1 00000100:00100000:1.0:1789760801.095402:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780183682448 time 780200112538 state 0x0 ref 1 00000100:00100000:1.0:1789760801.095415:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508513024:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.095461:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26390528, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.095471:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5773c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.095485:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.095490:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.095496:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5773c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.095510:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5773c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.110870:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973763 is committed 00002000:00010000:1.0:1789760801.110872:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26390528 -> 26394624 00010000:00010000:1.0:1789760801.110890:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5773c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.110904:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5773c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.110914:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5773c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.110922:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.110926:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5773c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.110935:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5773c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.110956:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5773c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.110964:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f28476e/0xb49177c0efe5773c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26390528->26394623] (req 26390528->26394623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.111054:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508513024:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15635us (15888us total) trans 4294973763 rc 0/0 00000100:00100000:1.0:1789760801.111069:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6475 00000100:00100000:1.0:1789760801.111080:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780183682448 time 780200112538 state 0x2 ref 0 00000100:00100000:1.0:1789760801.113714:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760801.114424:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508513152 00000100:00100000:0.0:1789760801.114435:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508513152 00000100:00100000:0.0:1789760801.114487:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780183682448 time 780200112538 state 0x0 ref 1 00000100:00100000:0.0:1789760801.114502:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780200372954 time 780200112538 state 0x1 ref 1 00000100:00100000:0.0:1789760801.114529:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780200372954 time 780219239219 state 0x0 ref 1 00000100:00100000:0.0:1789760801.114542:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508513152:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760801.114589:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26394624, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:0.0:1789760801.114598:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57743 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.114614:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760801.114618:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760801.114625:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57743 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.114651:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57743 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760801.130828:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973764 is committed 00002000:00010000:0.0:1789760801.131226:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26394624 -> 26398720 00010000:00010000:0.0:1789760801.131259:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57743 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.131272:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57743 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.131281:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57743 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.131313:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760801.131316:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57743 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.131323:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57743 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.131340:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57743 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.131345:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57743 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26394624->26398719] (req 26394624->26398719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760801.131431:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508513152:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16885us (17719us total) trans 4294973764 rc 0/0 00000100:00100000:0.0:1789760801.131442:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6476 00000100:00100000:0.0:1789760801.131453:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780200372954 time 780219239219 state 0x2 ref 0 00000100:00100000:3.0:1789760801.135034:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760801.135107:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508513280 00000100:00100000:0.0:1789760801.135117:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508513280 00000100:00100000:0.0:1789760801.135169:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780200372954 time 780219239219 state 0x0 ref 1 00000100:00100000:0.0:1789760801.135184:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780219499635 time 780219239219 state 0x1 ref 1 00000100:00100000:0.0:1789760801.135218:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780219499635 time 780239928079 state 0x0 ref 1 00000100:00100000:0.0:1789760801.135231:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508513280:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760801.135280:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26398720, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:0.0:1789760801.135315:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5774a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.135331:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760801.135336:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760801.135342:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5774a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.135357:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5774a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.148587:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973765 is committed 00002000:00010000:0.0:1789760801.148609:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26398720 -> 26402816 00010000:00010000:0.0:1789760801.148624:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5774a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.148651:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5774a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.148662:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5774a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.148670:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760801.148674:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5774a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.148686:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5774a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.148705:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5774a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.148712:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe5774a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26398720->26402815] (req 26398720->26402815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760801.148793:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508513280:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13560us (13763us total) trans 4294973765 rc 0/0 00000100:00100000:0.0:1789760801.148802:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6477 00000100:00100000:0.0:1789760801.148812:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780219499635 time 780239928079 state 0x2 ref 0 00000100:00100000:3.0:1789760801.150740:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.150863:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508513408 00000100:00100000:1.0:1789760801.150876:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508513408 00000100:00100000:1.0:1789760801.150950:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780219499635 time 780239928079 state 0x0 ref 1 00000100:00100000:1.0:1789760801.150970:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780240188495 time 780239928079 state 0x1 ref 1 00000100:00100000:1.0:1789760801.151043:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780240188495 time 780255752261 state 0x0 ref 1 00000100:00100000:1.0:1789760801.151058:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508513408:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.151117:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26402816, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.151139:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57751 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.151158:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.151163:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.151170:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57751 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.151195:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57751 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.165894:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973766 is committed 00002000:00010000:3.0:1789760801.166007:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26402816 -> 26406912 00010000:00010000:3.0:1789760801.166022:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57751 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.166035:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57751 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.166044:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57751 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.166051:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.166056:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57751 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.166064:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57751 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.166083:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57751 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.166091:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7932028/0xb49177c0efe57751 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26402816->26406911] (req 26402816->26406911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.166189:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508513408:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15130us (15458us total) trans 4294973766 rc 0/0 00000100:00100000:3.0:1789760801.166212:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6478 00000100:00100000:3.0:1789760801.166223:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780240188495 time 780255752261 state 0x2 ref 0 00000100:00100000:1.0:1789760801.168811:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760801.168869:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508513536 00000100:00100000:0.0:1789760801.168879:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508513536 00000100:00100000:0.0:1789760801.168929:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780240188495 time 780255752261 state 0x0 ref 1 00000100:00100000:0.0:1789760801.168944:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780256012677 time 780255752261 state 0x1 ref 1 00000100:00100000:0.0:1789760801.168978:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780256012677 time 780273688357 state 0x0 ref 1 00000100:00100000:0.0:1789760801.168990:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508513536:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760801.169034:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26406912, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:0.0:1789760801.169042:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57758 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.169055:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760801.169059:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760801.169065:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57758 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.169078:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57758 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.180977:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973767 is committed 00002000:00010000:3.0:1789760801.181204:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26406912 -> 26411008 00010000:00010000:3.0:1789760801.181224:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57758 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.181236:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57758 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.181245:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57758 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.181250:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.181255:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57758 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.181262:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57758 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.181282:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57758 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.181322:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57758 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26406912->26411007] (req 26406912->26411007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.181451:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049beaf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508513536:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12457us (12641us total) trans 4294973767 rc 0/0 00000100:00100000:3.0:1789760801.181471:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6479 00000100:00100000:3.0:1789760801.181485:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780256012677 time 780273688357 state 0x2 ref 0 00000100:00100000:0.0:1789760801.183329:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.183366:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508513664 00000100:00100000:3.0:1789760801.183379:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508513664 00000100:00100000:3.0:1789760801.183443:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780256012677 time 780273688357 state 0x0 ref 1 00000100:00100000:3.0:1789760801.183461:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780273948773 time 780273688357 state 0x1 ref 1 00000100:00100000:3.0:1789760801.183500:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780273948773 time 780288210198 state 0x0 ref 1 00000100:00100000:3.0:1789760801.183514:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508513664:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.183572:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26411008, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760801.183582:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5775f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.183599:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760801.183603:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.183611:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5775f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.183640:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5775f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.200167:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973768 is committed 00002000:00010000:1.0:1789760801.200227:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26411008 -> 26415104 00010000:00010000:1.0:1789760801.200243:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5775f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.200255:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5775f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.200265:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5775f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.200272:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.200276:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5775f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.200299:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5775f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.200315:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5775f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.200320:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adfa3cfb/0xb49177c0efe5775f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26411008->26415103] (req 26411008->26415103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.200396:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508513664:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16877us (17070us total) trans 4294973768 rc 0/0 00000100:00100000:1.0:1789760801.200408:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6480 00000100:00100000:1.0:1789760801.200416:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780273948773 time 780288210198 state 0x2 ref 0 00000100:00100000:0.0:1789760801.204163:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.204209:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508513792 00000100:00100000:3.0:1789760801.204218:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508513792 00000100:00100000:3.0:1789760801.204320:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780273948773 time 780288210198 state 0x0 ref 1 00000100:00100000:3.0:1789760801.204333:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780288470614 time 780288210198 state 0x1 ref 1 00000100:00100000:3.0:1789760801.204357:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780288470614 time 780309068856 state 0x0 ref 1 00000100:00100000:3.0:1789760801.204367:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508513792:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.204412:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26415104, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760801.204421:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57766 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.204436:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760801.204440:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.204446:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57766 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.204459:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57766 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.216904:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973769 is committed 00002000:00010000:1.0:1789760801.216934:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26415104 -> 26419200 00010000:00010000:1.0:1789760801.216947:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57766 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.216956:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57766 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.216962:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57766 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.216967:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.216970:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57766 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.216975:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57766 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.216989:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57766 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.216994:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb0d00e6/0xb49177c0efe57766 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26415104->26419199] (req 26415104->26419199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.217078:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508513792:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12708us (12919us total) trans 4294973769 rc 0/0 00000100:00100000:1.0:1789760801.217087:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6481 00000100:00100000:1.0:1789760801.217095:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780288470614 time 780309068856 state 0x2 ref 0 00000100:00100000:2.0:1789760801.221550:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.221655:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508513920 00000100:00100000:3.0:1789760801.221664:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508513920 00000100:00100000:3.0:1789760801.221716:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780288470614 time 780309068856 state 0x0 ref 1 00000100:00100000:3.0:1789760801.221731:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780309329272 time 780309068856 state 0x1 ref 1 00000100:00100000:3.0:1789760801.221757:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780309329272 time 780326466951 state 0x0 ref 1 00000100:00100000:3.0:1789760801.221769:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508513920:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.221813:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26419200, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760801.221822:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5776d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.221837:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760801.221842:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.221848:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5776d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.221861:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5776d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760801.237956:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973770 is committed 00002000:00010000:3.0:1789760801.237969:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26419200 -> 26423296 00010000:00010000:3.0:1789760801.238050:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5776d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.238128:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5776d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.238138:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5776d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.238146:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.238151:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5776d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.238162:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5776d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.238183:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5776d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.238191:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe5776d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26419200->26423295] (req 26419200->26423295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.238276:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508513920:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16503us (16729us total) trans 4294973770 rc 0/0 00000100:00100000:3.0:1789760801.238316:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6482 00000100:00100000:3.0:1789760801.238326:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780309329272 time 780326466951 state 0x2 ref 0 00000100:00100000:0.0:1789760801.240074:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.240144:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508514048 00000100:00100000:1.0:1789760801.240154:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508514048 00000100:00100000:1.0:1789760801.240202:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780309329272 time 780326466951 state 0x0 ref 1 00000100:00100000:1.0:1789760801.240238:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780326727367 time 780326466951 state 0x1 ref 1 00000100:00100000:1.0:1789760801.240262:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780326727367 time 780344973289 state 0x0 ref 1 00000100:00100000:1.0:1789760801.240275:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508514048:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.240350:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26423296, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.240358:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57774 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.240372:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.240377:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.240383:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57774 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.240397:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57774 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.252073:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973771 is committed 00002000:00010000:1.0:1789760801.252112:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26423296 -> 26427392 00010000:00010000:1.0:1789760801.252132:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57774 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.252145:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57774 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.252154:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57774 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.252161:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.252166:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57774 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.252175:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57774 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.252195:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57774 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.252202:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57774 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26423296->26427391] (req 26423296->26427391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.252343:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508514048:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12065us (12269us total) trans 4294973771 rc 0/0 00000100:00100000:1.0:1789760801.252357:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6483 00000100:00100000:1.0:1789760801.252366:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780326727367 time 780344973289 state 0x2 ref 0 00000100:00100000:0.0:1789760801.255477:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.255633:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508514176 00000100:00100000:1.0:1789760801.255647:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508514176 00000100:00100000:1.0:1789760801.255738:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780326727367 time 780344973289 state 0x0 ref 1 00000100:00100000:1.0:1789760801.255757:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780345233705 time 780344973289 state 0x1 ref 1 00000100:00100000:1.0:1789760801.255784:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780345233705 time 780360494580 state 0x0 ref 1 00000100:00100000:1.0:1789760801.255797:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508514176:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.255867:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26427392, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.255877:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5777b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.255895:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.255907:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.255915:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5777b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.255939:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5777b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.271503:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973772 is committed 00002000:00010000:1.0:1789760801.271534:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26427392 -> 26431488 00010000:00010000:1.0:1789760801.271555:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5777b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.271569:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5777b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.271578:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5777b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.271585:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.271590:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5777b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.271598:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5777b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.271636:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5777b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.271644:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe5777b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26427392->26431487] (req 26427392->26431487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.271729:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508514176:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15929us (16262us total) trans 4294973772 rc 0/0 00000100:00100000:1.0:1789760801.271739:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6484 00000100:00100000:1.0:1789760801.271748:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780345233705 time 780360494580 state 0x2 ref 0 00000100:00100000:2.0:1789760801.274490:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.274725:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508514304 00000100:00100000:3.0:1789760801.274743:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508514304 00000100:00100000:3.0:1789760801.274831:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780345233705 time 780360494580 state 0x0 ref 1 00000100:00100000:3.0:1789760801.274851:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780360754996 time 780360494580 state 0x1 ref 1 00000100:00100000:3.0:1789760801.274900:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780360754996 time 780379610118 state 0x0 ref 1 00000100:00100000:3.0:1789760801.274916:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508514304:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.274999:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26431488, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760801.275012:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57782 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.275030:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760801.275036:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.275043:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57782 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.275059:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57782 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.289728:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973773 is committed 00002000:00010000:3.0:1789760801.289743:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26431488 -> 26435584 00010000:00010000:3.0:1789760801.289757:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57782 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.289770:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57782 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.289780:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57782 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.289787:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.289792:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57782 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.289800:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57782 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.289819:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57782 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.289827:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57782 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26431488->26435583] (req 26431488->26435583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.289912:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508514304:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14995us (15426us total) trans 4294973773 rc 0/0 00000100:00100000:3.0:1789760801.289923:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6485 00000100:00100000:3.0:1789760801.289933:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780360754996 time 780379610118 state 0x2 ref 0 00000100:00100000:0.0:1789760801.291768:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.291813:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508514432 00000100:00100000:1.0:1789760801.291821:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508514432 00000100:00100000:1.0:1789760801.291859:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780360754996 time 780379610118 state 0x0 ref 1 00000100:00100000:1.0:1789760801.291869:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780379870534 time 780379610118 state 0x1 ref 1 00000100:00100000:1.0:1789760801.291887:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780379870534 time 780396598555 state 0x0 ref 1 00000100:00100000:1.0:1789760801.291896:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508514432:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.291937:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26435584, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.291944:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57789 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.291954:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.291957:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.291961:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57789 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.291970:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57789 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.304810:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973774 is committed 00002000:00010000:1.0:1789760801.304849:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26435584 -> 26439680 00010000:00010000:1.0:1789760801.304865:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57789 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.304877:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57789 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.304891:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57789 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.304899:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.304904:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57789 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.304913:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57789 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.304947:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57789 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.304955:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57789 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26435584->26439679] (req 26435584->26439679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.305047:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508514432:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13146us (13276us total) trans 4294973774 rc 0/0 00000100:00100000:1.0:1789760801.305058:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6486 00000100:00100000:1.0:1789760801.305071:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780379870534 time 780396598555 state 0x2 ref 0 00000100:00100000:0.0:1789760801.307397:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.307446:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508514560 00000100:00100000:1.0:1789760801.307456:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508514560 00000100:00100000:1.0:1789760801.307506:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780379870534 time 780396598555 state 0x0 ref 1 00000100:00100000:1.0:1789760801.307522:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780396858971 time 780396598555 state 0x1 ref 1 00000100:00100000:1.0:1789760801.307547:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780396858971 time 780412257769 state 0x0 ref 1 00000100:00100000:1.0:1789760801.307560:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508514560:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.307697:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26439680, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.307705:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57790 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.307720:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.307724:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.307730:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57790 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.307744:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57790 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.320884:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973775 is committed 00002000:00010000:1.0:1789760801.320961:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26439680 -> 26443776 00010000:00010000:1.0:1789760801.320976:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57790 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.320990:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57790 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.320999:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57790 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.321006:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.321011:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57790 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.321019:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57790 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.321039:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57790 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.321047:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57790 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26439680->26443775] (req 26439680->26443775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.321118:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508514560:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13557us (13725us total) trans 4294973775 rc 0/0 00000100:00100000:1.0:1789760801.321126:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6487 00000100:00100000:1.0:1789760801.321134:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780396858971 time 780412257769 state 0x2 ref 0 00000100:00100000:2.0:1789760801.324457:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.324594:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508514688 00000100:00100000:3.0:1789760801.324611:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508514688 00000100:00100000:3.0:1789760801.324701:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780396858971 time 780412257769 state 0x0 ref 1 00000100:00100000:3.0:1789760801.324720:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780412518185 time 780412257769 state 0x1 ref 1 00000100:00100000:3.0:1789760801.324755:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780412518185 time 780429465496 state 0x0 ref 1 00000100:00100000:3.0:1789760801.324772:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508514688:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.324835:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26443776, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:3.0:1789760801.324847:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57797 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.324866:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760801.324872:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.324880:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57797 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.324897:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57797 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.342427:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973776 is committed 00002000:00010000:1.0:1789760801.342486:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26443776 -> 26447872 00010000:00010000:1.0:1789760801.342511:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57797 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.342523:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57797 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.342532:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57797 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.342543:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.342548:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57797 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.342565:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57797 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.342584:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57797 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.342590:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000463e1e7f/0xb49177c0efe57797 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26443776->26447871] (req 26443776->26447871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.342702:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508514688:12345-192.168.203.23@tcp:4:dd.0 Request processed in 17929us (18248us total) trans 4294973776 rc 0/0 00000100:00100000:1.0:1789760801.342713:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6488 00000100:00100000:1.0:1789760801.342722:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780412518185 time 780429465496 state 0x2 ref 0 00000100:00100000:0.0:1789760801.345039:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.345117:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508514816 00000100:00100000:1.0:1789760801.345126:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508514816 00000100:00100000:1.0:1789760801.345173:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780412518185 time 780429465496 state 0x0 ref 1 00000100:00100000:1.0:1789760801.345187:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780429725912 time 780429465496 state 0x1 ref 1 00000100:00100000:1.0:1789760801.345207:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780429725912 time 780449918804 state 0x0 ref 1 00000100:00100000:1.0:1789760801.345218:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508514816:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.345255:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26447872, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.345262:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe5779e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.345277:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.345281:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.345348:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe5779e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.345361:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe5779e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.364822:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973777 is committed 00002000:00010000:1.0:1789760801.364941:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26447872 -> 26451968 00010000:00010000:1.0:1789760801.364957:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe5779e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.364969:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe5779e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.364978:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe5779e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.364985:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.364998:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe5779e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.365008:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe5779e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.365029:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe5779e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.365037:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe5779e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26447872->26451967] (req 26447872->26451967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.365131:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508514816:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19905us (20090us total) trans 4294973777 rc 0/0 00000100:00100000:1.0:1789760801.365142:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6489 00000100:00100000:1.0:1789760801.365152:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780429725912 time 780449918804 state 0x2 ref 0 00000100:00100000:0.0:1789760801.367326:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.367360:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508514944 00000100:00100000:1.0:1789760801.367370:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508514944 00000100:00100000:1.0:1789760801.367422:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780429725912 time 780449918804 state 0x0 ref 1 00000100:00100000:1.0:1789760801.367447:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780450179220 time 780449918804 state 0x1 ref 1 00000100:00100000:1.0:1789760801.367478:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780450179220 time 780472188464 state 0x0 ref 1 00000100:00100000:1.0:1789760801.367491:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04984e7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508514944:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.367547:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26451968, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.367556:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.367571:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.367575:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.367582:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.367597:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789760801.385570:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26451968 -> 26456064 00010000:00010000:1.0:1789760801.385587:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.385599:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.385606:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.385614:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.385618:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.385636:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.385655:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.385663:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26451968->26456063] (req 26451968->26456063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.385669:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973778 is committed 00000100:00100000:1.0:1789760801.385748:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04984e7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508514944:12345-192.168.203.23@tcp:4:dd.0 Request processed in 18254us (18423us total) trans 4294973778 rc 0/0 00000100:00100000:1.0:1789760801.385759:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6490 00000100:00100000:1.0:1789760801.385769:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780450179220 time 780472188464 state 0x2 ref 0 00000100:00100000:2.0:1789760801.387759:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.387822:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508515072 00000100:00100000:1.0:1789760801.387831:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508515072 00000100:00100000:1.0:1789760801.387882:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780450179220 time 780472188464 state 0x0 ref 1 00000100:00100000:1.0:1789760801.387910:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780472448880 time 780472188464 state 0x1 ref 1 00000100:00100000:1.0:1789760801.387948:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780472448880 time 780492649965 state 0x0 ref 1 00000100:00100000:1.0:1789760801.387960:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508515072:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.388000:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26456064, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.388008:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.388024:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.388028:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.388034:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.388056:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.401882:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973779 is committed 00002000:00010000:1.0:1789760801.401913:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26456064 -> 26460160 00010000:00010000:1.0:1789760801.401928:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.401937:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.401947:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.401954:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.401958:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.401967:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.401987:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.401995:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26456064->26460159] (req 26456064->26460159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.402081:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508515072:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14119us (14323us total) trans 4294973779 rc 0/0 00000100:00100000:1.0:1789760801.402089:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6491 00000100:00100000:1.0:1789760801.402096:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780472448880 time 780492649965 state 0x2 ref 0 00000100:00100000:0.0:1789760801.404233:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.404310:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508515200 00000100:00100000:1.0:1789760801.404320:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508515200 00000100:00100000:1.0:1789760801.404385:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780472448880 time 780492649965 state 0x0 ref 1 00000100:00100000:1.0:1789760801.404399:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780492910381 time 780492649965 state 0x1 ref 1 00000100:00100000:1.0:1789760801.404429:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780492910381 time 780509139100 state 0x0 ref 1 00000100:00100000:1.0:1789760801.404448:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508515200:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.404492:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26460160, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.404500:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.404515:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.404520:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.404526:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.404552:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.419508:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973780 is committed 00002000:00010000:1.0:1789760801.419587:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26460160 -> 26464256 00010000:00010000:1.0:1789760801.419602:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.419614:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.419636:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.419644:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.419649:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.419657:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.419675:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.419682:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26460160->26464255] (req 26460160->26464255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.419766:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508515200:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15321us (15536us total) trans 4294973780 rc 0/0 00000100:00100000:1.0:1789760801.419781:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6492 00000100:00100000:1.0:1789760801.419790:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780492910381 time 780509139100 state 0x2 ref 0 00000100:00100000:0.0:1789760801.422090:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.422152:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508515328 00000100:00100000:1.0:1789760801.422161:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508515328 00000100:00100000:1.0:1789760801.422201:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780492910381 time 780509139100 state 0x0 ref 1 00000100:00100000:1.0:1789760801.422211:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780509399516 time 780509139100 state 0x1 ref 1 00000100:00100000:1.0:1789760801.422231:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780509399516 time 780526943314 state 0x0 ref 1 00000100:00100000:1.0:1789760801.422240:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508515328:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.422272:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26464256, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.422277:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.422307:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.422310:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.422315:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.422326:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.437184:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973781 is committed 00002000:00010000:1.0:1789760801.437322:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26464256 -> 26468352 00010000:00010000:1.0:1789760801.437335:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.437350:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.437358:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.437365:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.437369:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.437378:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.437398:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.437405:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26464256->26468351] (req 26464256->26468351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.437499:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c2dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508515328:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15254us (15408us total) trans 4294973781 rc 0/0 00000100:00100000:1.0:1789760801.437509:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6493 00000100:00100000:1.0:1789760801.437518:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780509399516 time 780526943314 state 0x2 ref 0 00000100:00100000:2.0:1789760801.440094:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.440160:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508515456 00000100:00100000:1.0:1789760801.440173:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508515456 00000100:00100000:1.0:1789760801.440231:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780509399516 time 780526943314 state 0x0 ref 1 00000100:00100000:1.0:1789760801.440245:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780527203730 time 780526943314 state 0x1 ref 1 00000100:00100000:1.0:1789760801.440278:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780527203730 time 780544987984 state 0x0 ref 1 00000100:00100000:1.0:1789760801.440317:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508515456:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.440366:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26468352, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.440377:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.440390:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.440394:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.440400:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.440414:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.456418:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973782 is committed 00002000:00010000:1.0:1789760801.456465:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26468352 -> 26472448 00010000:00010000:1.0:1789760801.456481:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.456494:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.456509:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.456517:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.456521:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.456530:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.456547:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.456555:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26468352->26472447] (req 26468352->26472447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.456641:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508515456:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16311us (16541us total) trans 4294973782 rc 0/0 00000100:00100000:1.0:1789760801.456651:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6494 00000100:00100000:1.0:1789760801.456661:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780527203730 time 780544987984 state 0x2 ref 0 00000100:00100000:0.0:1789760801.458924:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.459086:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508515584 00000100:00100000:1.0:1789760801.459096:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508515584 00000100:00100000:1.0:1789760801.459150:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780527203730 time 780544987984 state 0x0 ref 1 00000100:00100000:1.0:1789760801.459166:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780545248400 time 780544987984 state 0x1 ref 1 00000100:00100000:1.0:1789760801.459204:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780545248400 time 780563914674 state 0x0 ref 1 00000100:00100000:1.0:1789760801.459216:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508515584:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.459259:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26472448, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.459268:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.459322:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.459328:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.459335:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.459362:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.473986:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973783 is committed 00002000:00010000:1.0:1789760801.474372:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26472448 -> 26476544 00010000:00010000:1.0:1789760801.474385:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.474393:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.474399:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.474403:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.474407:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.474414:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.474428:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.474432:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26472448->26476543] (req 26472448->26476543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.474514:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508515584:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15295us (15592us total) trans 4294973783 rc 0/0 00000100:00100000:1.0:1789760801.474523:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6495 00000100:00100000:1.0:1789760801.474533:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780545248400 time 780563914674 state 0x2 ref 0 00000100:00100000:0.0:1789760801.477189:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.477272:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508515712 00000100:00100000:1.0:1789760801.477281:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508515712 00000100:00100000:1.0:1789760801.477416:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780545248400 time 780563914674 state 0x0 ref 1 00000100:00100000:1.0:1789760801.477442:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780564175090 time 780563914674 state 0x1 ref 1 00000100:00100000:1.0:1789760801.477471:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780564175090 time 780582178399 state 0x0 ref 1 00000100:00100000:1.0:1789760801.477483:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508515712:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.477530:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26476544, mtime: 0x6aad9521, atime: 0x0, ctime: 0x6aad9521, blocks: 0xd009 00010000:00010000:1.0:1789760801.477538:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.477552:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.477556:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.477570:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.477585:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.495754:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973784 is committed 00002000:00010000:1.0:1789760801.495775:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26476544 -> 26480640 00002000:00010000:1.0:1789760801.495779:0:8109:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb mtime from disk: 1789760801 -> 1789760802 00002000:00010000:1.0:1789760801.495781:0:8109:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb ctime from disk: 1789760801 -> 1789760802 00010000:00010000:1.0:1789760801.495792:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.495801:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.495807:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.495811:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.495814:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.495824:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.495838:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.495843:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26476544->26480639] (req 26476544->26480639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.495912:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508515712:12345-192.168.203.23@tcp:4:dd.0 Request processed in 18429us (18728us total) trans 4294973784 rc 0/0 00000100:00100000:1.0:1789760801.495921:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6496 00000100:00100000:1.0:1789760801.495928:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780564175090 time 780582178399 state 0x2 ref 0 00000100:00100000:2.0:1789760801.499970:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.500027:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508515840 00000100:00100000:1.0:1789760801.500037:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508515840 00000100:00100000:1.0:1789760801.500084:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780564175090 time 780582178399 state 0x0 ref 1 00000100:00100000:1.0:1789760801.500096:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780582438815 time 780582178399 state 0x1 ref 1 00000100:00100000:1.0:1789760801.500116:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780582438815 time 780604828190 state 0x0 ref 1 00000100:00100000:1.0:1789760801.500127:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508515840:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.500179:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26480640, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.500188:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.500204:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.500207:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.500212:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.500223:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.512481:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973785 is committed 00002000:00010000:1.0:1789760801.512501:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26480640 -> 26484736 00010000:00010000:1.0:1789760801.512516:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.512529:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.512538:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.512545:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.512550:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.512558:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.512579:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.512588:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26480640->26484735] (req 26480640->26484735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.512692:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508515840:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12560us (12724us total) trans 4294973785 rc 0/0 00000100:00100000:1.0:1789760801.512702:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6497 00000100:00100000:1.0:1789760801.512713:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780582438815 time 780604828190 state 0x2 ref 0 00000100:00100000:0.0:1789760801.515511:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.515549:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508515968 00000100:00100000:1.0:1789760801.515559:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508515968 00000100:00100000:1.0:1789760801.515610:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780582438815 time 780604828190 state 0x0 ref 1 00000100:00100000:1.0:1789760801.515620:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780605088606 time 780604828190 state 0x1 ref 1 00000100:00100000:1.0:1789760801.515658:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780605088606 time 780620368124 state 0x0 ref 1 00000100:00100000:1.0:1789760801.515679:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508515968:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.515722:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26484736, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.515729:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.515741:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.515744:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.515748:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.515763:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.530828:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973786 is committed 00002000:00010000:1.0:1789760801.530858:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26484736 -> 26488832 00010000:00010000:1.0:1789760801.530871:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.530884:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.530893:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.530900:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.530904:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.530913:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.530943:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.530950:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26484736->26488831] (req 26484736->26488831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.531038:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0498546a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508515968:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15356us (15526us total) trans 4294973786 rc 0/0 00000100:00100000:1.0:1789760801.531048:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6498 00000100:00100000:1.0:1789760801.531057:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780605088606 time 780620368124 state 0x2 ref 0 00010000:00010000:3.0:1789760801.534403:0:8119:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760801.537005:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.537062:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508516096 00000100:00100000:1.0:1789760801.537072:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508516096 00000100:00100000:1.0:1789760801.537125:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780605088606 time 780620368124 state 0x0 ref 1 00000100:00100000:1.0:1789760801.537140:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780620628540 time 780620368124 state 0x1 ref 1 00000100:00100000:1.0:1789760801.537168:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780620628540 time 780641878545 state 0x0 ref 1 00000100:00100000:1.0:1789760801.537181:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508516096:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.537226:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26488832, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.537235:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.537250:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.537255:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.537261:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.537275:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.552526:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973787 is committed 00002000:00010000:1.0:1789760801.552579:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26488832 -> 26492928 00010000:00010000:1.0:1789760801.552596:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.552609:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.552634:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.552641:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.552646:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.552656:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.552691:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.552699:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26488832->26492927] (req 26488832->26492927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.552782:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508516096:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15599us (15781us total) trans 4294973787 rc 0/0 00000100:00100000:1.0:1789760801.552796:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6499 00000100:00100000:1.0:1789760801.552810:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780620628540 time 780641878545 state 0x2 ref 0 00000100:00100000:2.0:1789760801.555681:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.556156:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508516224 00000100:00100000:1.0:1789760801.556166:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508516224 00000100:00100000:1.0:1789760801.556214:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780620628540 time 780641878545 state 0x0 ref 1 00000100:00100000:1.0:1789760801.556234:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780642138961 time 780641878545 state 0x1 ref 1 00000100:00100000:1.0:1789760801.556256:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780642138961 time 780660967741 state 0x0 ref 1 00000100:00100000:1.0:1789760801.556267:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508516224:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.556338:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26492928, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.556360:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.556375:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.556379:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.556385:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.556400:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.569091:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973788 is committed 00002000:00010000:1.0:1789760801.569146:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26492928 -> 26497024 00010000:00010000:1.0:1789760801.569162:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.569175:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.569184:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.569189:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.569193:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.569199:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.569213:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.569218:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26492928->26497023] (req 26492928->26497023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.569328:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508516224:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13020us (13612us total) trans 4294973788 rc 0/0 00000100:00100000:1.0:1789760801.569339:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6500 00000100:00100000:1.0:1789760801.569350:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780642138961 time 780660967741 state 0x2 ref 0 00000100:00100000:0.0:1789760801.571347:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.571430:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508516352 00000100:00100000:1.0:1789760801.571440:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508516352 00000100:00100000:1.0:1789760801.571487:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780642138961 time 780660967741 state 0x0 ref 1 00000100:00100000:1.0:1789760801.571501:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780661228157 time 780660967741 state 0x1 ref 1 00000100:00100000:1.0:1789760801.571532:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780661228157 time 780676242584 state 0x0 ref 1 00000100:00100000:1.0:1789760801.571545:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508516352:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.571598:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26497024, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.571607:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.571635:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.571640:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.571647:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.571661:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.586104:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973789 is committed 00002000:00010000:3.0:1789760801.586119:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26497024 -> 26501120 00010000:00010000:3.0:1789760801.586135:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.586149:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.586158:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.586165:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.586170:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.586183:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.586200:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.586205:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe577f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26497024->26501119] (req 26497024->26501119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.586272:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985468380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508516352:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14726us (14927us total) trans 4294973789 rc 0/0 00000100:00100000:3.0:1789760801.586279:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6501 00000100:00100000:3.0:1789760801.586316:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780661228157 time 780676242584 state 0x2 ref 0 00000100:00100000:3.0:1789760801.589522:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.589593:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508516480 00000100:00100000:1.0:1789760801.589603:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508516480 00000100:00100000:1.0:1789760801.589670:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780661228157 time 780676242584 state 0x0 ref 1 00000100:00100000:1.0:1789760801.589691:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780676503000 time 780676242584 state 0x1 ref 1 00000100:00100000:1.0:1789760801.589719:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780676503000 time 780694429049 state 0x0 ref 1 00000100:00100000:1.0:1789760801.589734:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508516480:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.589788:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26501120, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.589796:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.589812:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.589817:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.589824:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.589836:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.602592:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973790 is committed 00002000:00010000:1.0:1789760801.602684:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26501120 -> 26505216 00010000:00010000:1.0:1789760801.602700:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.602714:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.602722:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.602729:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.602733:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.602742:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.602761:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.602769:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe577f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26501120->26505215] (req 26501120->26505215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.602898:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508516480:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13161us (13375us total) trans 4294973790 rc 0/0 00000100:00100000:1.0:1789760801.602908:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6502 00000100:00100000:1.0:1789760801.602918:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780676503000 time 780694429049 state 0x2 ref 0 00000100:00100000:2.0:1789760801.606198:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.606246:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508516608 00000100:00100000:1.0:1789760801.606256:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508516608 00000100:00100000:1.0:1789760801.606333:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780676503000 time 780694429049 state 0x0 ref 1 00000100:00100000:1.0:1789760801.606347:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780694689465 time 780694429049 state 0x1 ref 1 00000100:00100000:1.0:1789760801.606373:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780694689465 time 780711084024 state 0x0 ref 1 00000100:00100000:1.0:1789760801.606387:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508516608:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.606430:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26505216, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.606438:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57800 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.606459:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.606464:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.606470:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57800 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.606483:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57800 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.618781:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973791 is committed 00002000:00010000:1.0:1789760801.618826:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26505216 -> 26509312 00010000:00010000:1.0:1789760801.618842:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57800 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.618855:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57800 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.618863:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57800 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.618870:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.618875:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57800 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.618885:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57800 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.618905:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57800 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.618915:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe57800 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26505216->26509311] (req 26505216->26509311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.619005:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508516608:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12616us (12808us total) trans 4294973791 rc 0/0 00000100:00100000:1.0:1789760801.619015:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6503 00000100:00100000:1.0:1789760801.619025:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780694689465 time 780711084024 state 0x2 ref 0 00000100:00100000:0.0:1789760801.622328:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.622431:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508516736 00000100:00100000:3.0:1789760801.622441:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508516736 00000100:00100000:3.0:1789760801.622491:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780694689465 time 780711084024 state 0x0 ref 1 00000100:00100000:3.0:1789760801.622505:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780711344440 time 780711084024 state 0x1 ref 1 00000100:00100000:3.0:1789760801.622536:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780711344440 time 780727246476 state 0x0 ref 1 00000100:00100000:3.0:1789760801.622549:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04985469500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508516736:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.622595:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26509312, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760801.622604:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57807 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.622619:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760801.622637:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.622643:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57807 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.622658:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57807 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.639139:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973792 is committed 00002000:00010000:1.0:1789760801.639145:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26509312 -> 26513408 00010000:00010000:1.0:1789760801.639159:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57807 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.639168:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57807 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.639175:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57807 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.639179:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.639182:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57807 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.639188:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57807 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.639205:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57807 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.639210:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000080bc42f1/0xb49177c0efe57807 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26509312->26513407] (req 26509312->26513407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.639283:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04985469500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508516736:12345-192.168.203.23@tcp:4:dd.0 Request processed in 16732us (16960us total) trans 4294973792 rc 0/0 00000100:00100000:1.0:1789760801.639364:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6504 00000100:00100000:1.0:1789760801.639373:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780711344440 time 780727246476 state 0x2 ref 0 00000100:00100000:3.0:1789760801.642313:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.642342:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508516864 00000100:00100000:1.0:1789760801.642352:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508516864 00000100:00100000:1.0:1789760801.642409:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780711344440 time 780727246476 state 0x0 ref 1 00000100:00100000:1.0:1789760801.642424:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780727506892 time 780727246476 state 0x1 ref 1 00000100:00100000:1.0:1789760801.642453:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780727506892 time 780747164011 state 0x0 ref 1 00000100:00100000:1.0:1789760801.642464:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508516864:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.642509:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26513408, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.642518:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5780e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.642533:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.642538:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.642544:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5780e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.642557:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5780e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.655444:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973793 is committed 00002000:00010000:3.0:1789760801.655544:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26513408 -> 26517504 00010000:00010000:3.0:1789760801.655556:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5780e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.655566:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5780e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.655580:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5780e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.655585:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56000 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.655588:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5780e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.655598:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5780e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.655612:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5780e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.655616:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd1af677/0xb49177c0efe5780e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26513408->26517503] (req 26513408->26517503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.655694:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508516864:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13229us (13414us total) trans 4294973793 rc 0/0 00000100:00100000:3.0:1789760801.655702:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6505 00000100:00100000:3.0:1789760801.655708:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780727506892 time 780747164011 state 0x2 ref 0 00000100:00100000:2.0:1789760801.658715:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.658785:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508516992 00000100:00100000:1.0:1789760801.658795:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508516992 00000100:00100000:1.0:1789760801.658845:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780727506892 time 780747164011 state 0x0 ref 1 00000100:00100000:1.0:1789760801.658861:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780747424427 time 780747164011 state 0x1 ref 1 00000100:00100000:1.0:1789760801.658893:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780747424427 time 780763604006 state 0x0 ref 1 00000100:00100000:1.0:1789760801.658906:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508516992:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.658953:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26517504, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.658962:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57815 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.658976:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.658980:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.658987:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57815 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.659002:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57815 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.672794:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973794 is committed 00002000:00010000:1.0:1789760801.672803:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26517504 -> 26521600 00010000:00010000:1.0:1789760801.672817:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57815 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.672827:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57815 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.672833:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57815 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.672838:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.672841:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57815 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.672848:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57815 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.672862:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57815 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.672866:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57815 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26517504->26521599] (req 26517504->26521599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.672952:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508516992:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14043us (14240us total) trans 4294973794 rc 0/0 00000100:00100000:1.0:1789760801.672966:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6506 00000100:00100000:1.0:1789760801.672976:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780747424427 time 780763604006 state 0x2 ref 0 00000100:00100000:0.0:1789760801.675428:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.675475:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508517120 00000100:00100000:1.0:1789760801.675485:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508517120 00000100:00100000:1.0:1789760801.675535:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780747424427 time 780763604006 state 0x0 ref 1 00000100:00100000:1.0:1789760801.675549:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780763864422 time 780763604006 state 0x1 ref 1 00000100:00100000:1.0:1789760801.675583:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780763864422 time 780780293979 state 0x0 ref 1 00000100:00100000:1.0:1789760801.675595:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508517120:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.675653:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26521600, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.675662:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5781c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.675677:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.675681:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.675687:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5781c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.675702:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5781c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.690903:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973795 is committed 00002000:00010000:0.0:1789760801.691046:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26521600 -> 26525696 00010000:00010000:0.0:1789760801.691062:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5781c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.691074:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5781c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.691083:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5781c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.691090:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760801.691094:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5781c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.691109:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5781c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.691128:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5781c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.691136:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000047059055/0xb49177c0efe5781c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26521600->26525695] (req 26521600->26525695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760801.691224:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508517120:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15625us (15798us total) trans 4294973795 rc 0/0 00000100:00100000:0.0:1789760801.691233:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6507 00000100:00100000:0.0:1789760801.691243:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780763864422 time 780780293979 state 0x2 ref 0 00000100:00100000:3.0:1789760801.693717:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.694254:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508517248 00000100:00100000:1.0:1789760801.694264:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508517248 00000100:00100000:1.0:1789760801.694349:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780763864422 time 780780293979 state 0x0 ref 1 00000100:00100000:1.0:1789760801.694364:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780780554395 time 780780293979 state 0x1 ref 1 00000100:00100000:1.0:1789760801.694391:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780780554395 time 780799101300 state 0x0 ref 1 00000100:00100000:1.0:1789760801.694404:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508517248:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.694459:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26525696, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.694468:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57823 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.694483:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.694487:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.694493:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57823 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.694506:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57823 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760801.711595:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973796 is committed 00002000:00010000:1.0:1789760801.711641:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26525696 -> 26529792 00010000:00010000:1.0:1789760801.711656:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57823 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.711668:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57823 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.711678:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57823 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.711685:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.711690:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57823 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.711698:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57823 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.711717:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57823 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.711723:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57823 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26525696->26529791] (req 26525696->26529791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.711811:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508517248:12345-192.168.203.23@tcp:4:dd.0 Request processed in 17406us (18098us total) trans 4294973796 rc 0/0 00000100:00100000:1.0:1789760801.711820:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6508 00000100:00100000:1.0:1789760801.711830:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780780554395 time 780799101300 state 0x2 ref 0 00000100:00100000:2.0:1789760801.714434:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.714479:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508517376 00000100:00100000:1.0:1789760801.714489:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508517376 00000100:00100000:1.0:1789760801.714538:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780780554395 time 780799101300 state 0x0 ref 1 00000100:00100000:1.0:1789760801.714558:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780799361716 time 780799101300 state 0x1 ref 1 00000100:00100000:1.0:1789760801.714588:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780799361716 time 780819297914 state 0x0 ref 1 00000100:00100000:1.0:1789760801.714611:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048851d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508517376:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.715320:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26529792, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.715330:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5782a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.715346:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.715351:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.715357:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5782a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.715379:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5782a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.729029:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973797 is committed 00002000:00010000:0.0:1789760801.729071:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26529792 -> 26533888 00010000:00010000:0.0:1789760801.729087:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5782a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.729099:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5782a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.729107:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5782a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.729114:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57e00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760801.729119:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5782a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.729126:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5782a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.729144:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5782a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.729151:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e4d9c9e8/0xb49177c0efe5782a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26529792->26533887] (req 26529792->26533887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760801.729236:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048851d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508517376:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14625us (14799us total) trans 4294973797 rc 0/0 00000100:00100000:0.0:1789760801.729247:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6509 00000100:00100000:0.0:1789760801.729257:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780799361716 time 780819297914 state 0x2 ref 0 00000100:00100000:0.0:1789760801.731823:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.731868:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508517504 00000100:00100000:1.0:1789760801.731876:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508517504 00000100:00100000:1.0:1789760801.731916:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780799361716 time 780819297914 state 0x0 ref 1 00000100:00100000:1.0:1789760801.731926:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780819558330 time 780819297914 state 0x1 ref 1 00000100:00100000:1.0:1789760801.731943:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780819558330 time 780836655271 state 0x0 ref 1 00000100:00100000:1.0:1789760801.731952:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508517504:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.731988:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26533888, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.731996:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57831 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.732016:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.732020:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.732027:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57831 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.732040:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57831 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760801.745377:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26533888 -> 26537984 00000001:00080000:3.0:1789760801.745387:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973798 is committed 00010000:00010000:2.0:1789760801.745396:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57831 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.745411:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57831 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.745420:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57831 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.745428:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760801.745433:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57831 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.745441:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57831 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.745460:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57831 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.745474:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57831 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26533888->26537983] (req 26533888->26537983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760801.745564:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508517504:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13609us (13742us total) trans 4294973798 rc 0/0 00000100:00100000:2.0:1789760801.745575:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6510 00000100:00100000:2.0:1789760801.745584:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780819558330 time 780836655271 state 0x2 ref 0 00000100:00100000:3.0:1789760801.747536:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.748012:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508517632 00000100:00100000:1.0:1789760801.748021:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508517632 00000100:00100000:1.0:1789760801.748066:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780819558330 time 780836655271 state 0x0 ref 1 00000100:00100000:1.0:1789760801.748081:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780836915687 time 780836655271 state 0x1 ref 1 00000100:00100000:1.0:1789760801.748118:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780836915687 time 780852828360 state 0x0 ref 1 00000100:00100000:1.0:1789760801.748132:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508517632:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.748178:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26537984, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.748186:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57838 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.748200:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.748204:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.748208:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57838 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.748223:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57838 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.760759:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973799 is committed 00002000:00010000:3.0:1789760801.760835:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26537984 -> 26542080 00010000:00010000:3.0:1789760801.760848:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57838 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.760857:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57838 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.760870:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57838 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.760878:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca56400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.760882:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57838 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.760891:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57838 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.760910:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57838 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.760926:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000f311d2c/0xb49177c0efe57838 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26537984->26542079] (req 26537984->26542079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.761009:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa048846d2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508517632:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12875us (13472us total) trans 4294973799 rc 0/0 00000100:00100000:3.0:1789760801.761025:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6511 00000100:00100000:3.0:1789760801.761035:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780836915687 time 780852828360 state 0x2 ref 0 00000100:00100000:1.0:1789760801.763491:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760801.763550:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508517760 00000100:00100000:0.0:1789760801.763560:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508517760 00000100:00100000:0.0:1789760801.763607:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780836915687 time 780852828360 state 0x0 ref 1 00000100:00100000:0.0:1789760801.763634:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780853088776 time 780852828360 state 0x1 ref 1 00000100:00100000:0.0:1789760801.763667:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780853088776 time 780868377649 state 0x0 ref 1 00000100:00100000:0.0:1789760801.763679:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499bfec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508517760:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760801.763724:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26542080, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:0.0:1789760801.763734:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5783f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.763748:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760801.763752:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760801.763758:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5783f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.763771:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5783f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.776334:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973800 is committed 00002000:00010000:0.0:1789760801.776394:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26542080 -> 26546176 00010000:00010000:0.0:1789760801.776408:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5783f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.776420:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5783f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.776428:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5783f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.776436:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760801.776440:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5783f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.776455:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5783f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.776474:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5783f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.776482:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e345d114/0xb49177c0efe5783f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26542080->26546175] (req 26542080->26546175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760801.776566:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499bfec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508517760:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12884us (13078us total) trans 4294973800 rc 0/0 00000100:00100000:0.0:1789760801.776576:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6512 00000100:00100000:0.0:1789760801.776586:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780853088776 time 780868377649 state 0x2 ref 0 00000100:00100000:0.0:1789760801.778196:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760801.778244:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508517888 00000100:00100000:2.0:1789760801.778264:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508517888 00000100:00100000:2.0:1789760801.778325:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780853088776 time 780868377649 state 0x0 ref 1 00000100:00100000:2.0:1789760801.778339:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780868638065 time 780868377649 state 0x1 ref 1 00000100:00100000:2.0:1789760801.778364:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780868638065 time 780883074936 state 0x0 ref 1 00000100:00100000:2.0:1789760801.778377:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa049be26ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508517888:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760801.778417:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26546176, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:2.0:1789760801.778424:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe57846 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.778438:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:2.0:1789760801.778443:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760801.778447:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe57846 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.778474:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe57846 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.787453:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973801 is committed 00002000:00010000:2.0:1789760801.787485:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26546176 -> 26550272 00010000:00010000:2.0:1789760801.787500:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe57846 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.787512:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe57846 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.787521:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe57846 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.787527:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:2.0:1789760801.787538:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe57846 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.787547:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe57846 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.787565:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe57846 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.787573:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe57846 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26546176->26550271] (req 26546176->26550271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760801.787674:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa049be26ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508517888:12345-192.168.203.23@tcp:4:dd.0 Request processed in 9295us (9476us total) trans 4294973801 rc 0/0 00000100:00100000:2.0:1789760801.787685:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6513 00000100:00100000:2.0:1789760801.787695:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780868638065 time 780883074936 state 0x2 ref 0 00000100:00100000:3.0:1789760801.789540:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.789605:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508518016 00000100:00100000:1.0:1789760801.789613:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508518016 00000100:00100000:1.0:1789760801.789670:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780868638065 time 780883074936 state 0x0 ref 1 00000100:00100000:1.0:1789760801.789684:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780883335352 time 780883074936 state 0x1 ref 1 00000100:00100000:1.0:1789760801.789709:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780883335352 time 780894419603 state 0x0 ref 1 00000100:00100000:1.0:1789760801.789722:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508518016:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.789758:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26550272, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.789764:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5784d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.789775:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.789777:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.789781:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5784d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.789794:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5784d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.798929:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973802 is committed 00002000:00010000:1.0:1789760801.798929:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26550272 -> 26554368 00010000:00010000:1.0:1789760801.798947:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5784d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.798960:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5784d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.798970:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5784d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.798978:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:1.0:1789760801.798982:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5784d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.798988:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5784d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.799004:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5784d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.799009:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b936349b/0xb49177c0efe5784d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26550272->26554367] (req 26550272->26554367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760801.799091:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508518016:12345-192.168.203.23@tcp:4:dd.0 Request processed in 9367us (9558us total) trans 4294973802 rc 0/0 00000100:00100000:1.0:1789760801.799105:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6514 00000100:00100000:1.0:1789760801.799115:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780883335352 time 780894419603 state 0x2 ref 0 00000100:00100000:1.0:1789760801.800913:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760801.800966:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508518144 00000100:00100000:0.0:1789760801.800975:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508518144 00000100:00100000:0.0:1789760801.801015:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780883335352 time 780894419603 state 0x0 ref 1 00000100:00100000:0.0:1789760801.801028:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780894680019 time 780894419603 state 0x1 ref 1 00000100:00100000:0.0:1789760801.801053:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780894680019 time 780905764364 state 0x0 ref 1 00000100:00100000:0.0:1789760801.801064:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508518144:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760801.801102:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26554368, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:0.0:1789760801.801110:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57854 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.801124:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760801.801128:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760801.801134:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57854 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.801146:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57854 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760801.811230:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973803 is committed 00002000:00010000:0.0:1789760801.811244:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26554368 -> 26558464 00010000:00010000:0.0:1789760801.811260:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57854 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.811274:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57854 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.811300:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57854 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.811307:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760801.811311:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57854 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.811317:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57854 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.811333:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57854 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.811338:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57854 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26554368->26558463] (req 26554368->26558463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760801.811427:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508518144:12345-192.168.203.23@tcp:4:dd.0 Request processed in 10360us (10515us total) trans 4294973803 rc 0/0 00000100:00100000:0.0:1789760801.811436:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6515 00000100:00100000:0.0:1789760801.811455:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780894680019 time 780905764364 state 0x2 ref 0 00000100:00100000:0.0:1789760801.813154:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760801.813198:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508518272 00000100:00100000:2.0:1789760801.813206:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508518272 00000100:00100000:2.0:1789760801.813259:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780894680019 time 780905764364 state 0x0 ref 1 00000100:00100000:2.0:1789760801.813273:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780906024780 time 780905764364 state 0x1 ref 1 00000100:00100000:2.0:1789760801.813323:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780906024780 time 780918033505 state 0x0 ref 1 00000100:00100000:2.0:1789760801.813337:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508518272:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760801.813376:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26558464, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:2.0:1789760801.813384:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5785b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.813399:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:2.0:1789760801.813404:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760801.813409:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5785b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.813422:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5785b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.823646:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973804 is committed 00002000:00010000:3.0:1789760801.823696:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26558464 -> 26562560 00010000:00010000:3.0:1789760801.823712:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5785b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.823730:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5785b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.823739:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5785b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.823746:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.823751:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5785b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.823760:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5785b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.823779:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5785b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.823786:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000586ad7/0xb49177c0efe5785b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26558464->26562559] (req 26558464->26562559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.823877:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508518272:12345-192.168.203.23@tcp:4:dd.0 Request processed in 10540us (10722us total) trans 4294973804 rc 0/0 00000100:00100000:3.0:1789760801.823887:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6516 00000100:00100000:3.0:1789760801.823897:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780906024780 time 780918033505 state 0x2 ref 0 00000100:00100000:3.0:1789760801.826100:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.826331:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508518400 00000100:00100000:1.0:1789760801.826342:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508518400 00000100:00100000:1.0:1789760801.826394:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780906024780 time 780918033505 state 0x0 ref 1 00000100:00100000:1.0:1789760801.826409:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780918293921 time 780918033505 state 0x1 ref 1 00000100:00100000:1.0:1789760801.826434:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780918293921 time 780931145197 state 0x0 ref 1 00000100:00100000:1.0:1789760801.826459:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508518400:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.826505:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26562560, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.826515:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57862 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.826529:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.826533:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.826539:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57862 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.826552:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57862 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760801.839875:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973805 is committed 00002000:00010000:0.0:1789760801.839902:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26562560 -> 26566656 00010000:00010000:0.0:1789760801.839919:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57862 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.839930:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57862 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.839935:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57862 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.839940:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760801.839943:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57862 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.839950:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57862 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.839968:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57862 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.839972:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dda5dc41/0xb49177c0efe57862 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26562560->26566655] (req 26562560->26566655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760801.840046:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508518400:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13590us (13951us total) trans 4294973805 rc 0/0 00000100:00100000:0.0:1789760801.840054:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6517 00000100:00100000:0.0:1789760801.840062:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780918293921 time 780931145197 state 0x2 ref 0 00000100:00100000:1.0:1789760801.842615:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760801.842680:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508518528 00000100:00100000:0.0:1789760801.842690:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508518528 00000100:00100000:0.0:1789760801.842742:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780918293921 time 780931145197 state 0x0 ref 1 00000100:00100000:0.0:1789760801.842754:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780931405613 time 780931145197 state 0x1 ref 1 00000100:00100000:0.0:1789760801.842781:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780931405613 time 780947491642 state 0x0 ref 1 00000100:00100000:0.0:1789760801.842794:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508518528:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760801.842840:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26566656, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:0.0:1789760801.842849:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57869 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.842865:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:0.0:1789760801.842869:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760801.842875:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57869 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760801.842886:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57869 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760801.856591:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.856636:0:6459:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508518656 00000100:00100000:0.0:1789760801.856637:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.856648:0:6459:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508518656 00000100:00100000:0.0:1789760801.856682:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760801.856703:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508518784 00000100:00100000:1.0:1789760801.856708:0:6459:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 495 00000100:00100000:0.0:1789760801.856714:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760801.856718:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508518784 00000100:00100000:1.0:1789760801.856722:0:6459:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04988ca8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2409:x1876699508518656:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760801.856765:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508518912 00000100:00100000:2.0:1789760801.856770:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508518912 00000100:00100000:0.0:1789760801.856793:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.856793:0:6459:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04988ca8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:52f18b2b-fe4e-42a7-8be9-7ff39063b49e+13:2409:x1876699508518656:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 74us (211us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760801.856804:0:6467:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508519168 00000100:00100000:1.0:1789760801.856806:0:6459:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 495 00000100:00100000:2.0:1789760801.856809:0:6467:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508519168 00000100:00100000:0.0:1789760801.856821:0:14736:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508519040 00000100:00100000:0.0:1789760801.856823:0:14736:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508519040 00000100:00100000:2.0:1789760801.856833:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 1009 00000100:00100000:2.0:1789760801.856842:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508518784:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760801.856851:0:14736:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 1010 00000100:00100000:0.0:1789760801.856860:0:14736:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508518912:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:2.0:1789760801.856885:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+8:2408:x1876699508518784:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 46us (248us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760801.856893:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 1009 00000100:00100000:2.0:1789760801.856908:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 1011 00000100:00100000:2.0:1789760801.856914:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508519168:12345-192.168.203.23@tcp:400:kworker.0 00000100:00100000:0.0:1789760801.856954:0:14736:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508518912:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 95us (271us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760801.856954:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2409:x1876699508519168:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 39us (240us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760801.856962:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 1011 00000100:00100000:0.0:1789760801.856975:0:14736:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 1010 00000100:00100000:2.0:1789760801.856986:0:6467:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.23@tcp, seq: 1012 00000100:00100000:2.0:1789760801.856995:0:6467:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508519040:12345-192.168.203.23@tcp:400:kworker.0 00000100:00080000:0.0:1789760801.857004:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:0.0:1789760801.857009:0:6462:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760801.857023:0:6467:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:d99325d3-20d4-4c8a-8c36-c763a8fc5617+5:2408:x1876699508519040:12345-192.168.203.23@tcp:400:kworker.0 Request processed in 30us (233us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760801.857030:0:6467:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.23@tcp, seq: 1012 00000001:00080000:2.0:1789760801.858084:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973806 is committed 00002000:00010000:3.0:1789760801.858137:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26566656 -> 26570752 00010000:00010000:3.0:1789760801.858152:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57869 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.858164:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57869 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.858184:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57869 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.858193:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980030c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.858198:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57869 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.858207:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57869 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.858226:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57869 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.858234:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000379338e5/0xb49177c0efe57869 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26566656->26570751] (req 26566656->26570751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.858349:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508518528:12345-192.168.203.23@tcp:4:dd.0 Request processed in 15547us (15730us total) trans 4294973806 rc 0/0 00000100:00100000:3.0:1789760801.858363:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6518 00000100:00100000:3.0:1789760801.858515:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780931405613 time 780947491642 state 0x2 ref 0 00000100:00100000:3.0:1789760801.860494:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:2.0:1789760801.860558:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508519296 00000100:00100000:2.0:1789760801.860567:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508519296 00000100:00100000:2.0:1789760801.860606:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780931405613 time 780947491642 state 0x0 ref 1 00000100:00100000:2.0:1789760801.860617:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780947752058 time 780947491642 state 0x1 ref 1 00000100:00100000:2.0:1789760801.860658:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780947752058 time 780965368497 state 0x0 ref 1 00000100:00100000:2.0:1789760801.860670:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508519296:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760801.860716:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26570752, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:2.0:1789760801.860724:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe57870 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.860740:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:2.0:1789760801.860744:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760801.860751:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe57870 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.860764:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe57870 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.872330:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973807 is committed 00002000:00010000:3.0:1789760801.872385:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26570752 -> 26574848 00010000:00010000:3.0:1789760801.872401:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe57870 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.872414:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe57870 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.872423:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe57870 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.872431:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c293800 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.872435:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe57870 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.872445:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe57870 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.872466:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe57870 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.872473:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c70e77ec/0xb49177c0efe57870 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26570752->26574847] (req 26570752->26574847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.872561:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508519296:12345-192.168.203.23@tcp:4:dd.0 Request processed in 11889us (12179us total) trans 4294973807 rc 0/0 00000100:00100000:3.0:1789760801.872572:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6519 00000100:00100000:3.0:1789760801.872581:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780947752058 time 780965368497 state 0x2 ref 0 00000100:00100000:0.0:1789760801.874816:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:1.0:1789760801.874887:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508519424 00000100:00100000:1.0:1789760801.874897:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508519424 00000100:00100000:1.0:1789760801.874948:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780947752058 time 780965368497 state 0x0 ref 1 00000100:00100000:1.0:1789760801.874964:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780965628913 time 780965368497 state 0x1 ref 1 00000100:00100000:1.0:1789760801.874997:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780965628913 time 780979707767 state 0x0 ref 1 00000100:00100000:1.0:1789760801.875010:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508519424:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:1.0:1789760801.875068:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26574848, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:1.0:1789760801.875077:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57877 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.875091:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:1.0:1789760801.875096:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760801.875103:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57877 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760801.875122:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57877 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.889033:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973808 is committed 00002000:00010000:3.0:1789760801.889073:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26574848 -> 26578944 00010000:00010000:3.0:1789760801.889090:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57877 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.889104:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57877 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.889113:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57877 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.889120:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499ca57400 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.889125:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57877 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.889133:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57877 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.889153:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57877 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.889161:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000559fae21/0xb49177c0efe57877 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26574848->26578943] (req 26574848->26578943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.889243:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508519424:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14232us (14430us total) trans 4294973808 rc 0/0 00000100:00100000:3.0:1789760801.889253:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6520 00000100:00100000:3.0:1789760801.889263:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780965628913 time 780979707767 state 0x2 ref 0 00000100:00100000:1.0:1789760801.892892:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.892956:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508519552 00000100:00100000:3.0:1789760801.892979:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508519552 00000100:00100000:3.0:1789760801.893043:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780965628913 time 780979707767 state 0x0 ref 1 00000100:00100000:3.0:1789760801.893058:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780979968183 time 780979707767 state 0x1 ref 1 00000100:00100000:3.0:1789760801.893086:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780979968183 time 780997796526 state 0x0 ref 1 00000100:00100000:3.0:1789760801.893099:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508519552:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.893144:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26578944, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760801.893153:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5787e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.893169:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=780s v=(0 0) 00010000:00010000:3.0:1789760801.893173:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.893179:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5787e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.893192:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5787e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760801.907032:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973809 is committed 00002000:00010000:3.0:1789760801.907117:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26578944 -> 26583040 00010000:00010000:3.0:1789760801.907131:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5787e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.907144:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5787e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.907153:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5787e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.907161:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.907165:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5787e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.907172:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5787e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.907192:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5787e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.907198:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5787e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26578944->26583039] (req 26578944->26583039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.907282:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508519552:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14181us (14406us total) trans 4294973809 rc 0/0 00000100:00100000:3.0:1789760801.907430:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6521 00000100:00100000:3.0:1789760801.907441:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780979968183 time 780997796526 state 0x2 ref 0 00000100:00100000:0.0:1789760801.910239:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.910399:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508519680 00000100:00100000:3.0:1789760801.910408:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508519680 00000100:00100000:3.0:1789760801.910459:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780979968183 time 780997796526 state 0x0 ref 1 00000100:00100000:3.0:1789760801.910473:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780998056942 time 780997796526 state 0x1 ref 1 00000100:00100000:3.0:1789760801.910496:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780998056942 time 781015206480 state 0x0 ref 1 00000100:00100000:3.0:1789760801.910507:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508519680:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.910553:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26583040, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760801.910562:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57885 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.910577:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760801.910581:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.910588:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57885 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.910602:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57885 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.922117:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973810 is committed 00002000:00010000:3.0:1789760801.922280:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26583040 -> 26587136 00010000:00010000:3.0:1789760801.922330:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57885 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.922350:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57885 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.922359:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57885 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.922367:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.922372:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57885 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.922383:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57885 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.922408:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57885 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.922416:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57885 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26583040->26587135] (req 26583040->26587135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.922537:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508519680:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12024us (12299us total) trans 4294973810 rc 0/0 00000100:00100000:3.0:1789760801.922549:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6522 00000100:00100000:3.0:1789760801.922562:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780998056942 time 781015206480 state 0x2 ref 0 00000100:00100000:0.0:1789760801.925464:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.925687:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508519808 00000100:00100000:3.0:1789760801.925702:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508519808 00000100:00100000:3.0:1789760801.925761:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 780998056942 time 781015206480 state 0x0 ref 1 00000100:00100000:3.0:1789760801.925777:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781015466896 time 781015206480 state 0x1 ref 1 00000100:00100000:3.0:1789760801.925808:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781015466896 time 781030518849 state 0x0 ref 1 00000100:00100000:3.0:1789760801.925822:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508519808:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.925867:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26587136, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760801.925876:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5788c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.925888:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760801.925892:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.925896:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5788c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.925907:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5788c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.939503:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973811 is committed 00002000:00010000:3.0:1789760801.939539:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26587136 -> 26591232 00010000:00010000:3.0:1789760801.939554:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5788c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.939567:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5788c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.939582:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5788c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.939589:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.939594:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5788c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.939603:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5788c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.939635:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5788c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.939643:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5788c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26587136->26591231] (req 26587136->26591231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.939725:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508519808:12345-192.168.203.23@tcp:4:dd.0 Request processed in 13901us (14268us total) trans 4294973811 rc 0/0 00000100:00100000:3.0:1789760801.939736:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6523 00000100:00100000:3.0:1789760801.939746:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781015466896 time 781030518849 state 0x2 ref 0 00000100:00100000:1.0:1789760801.941714:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.941761:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508519936 00000100:00100000:3.0:1789760801.941780:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508519936 00000100:00100000:3.0:1789760801.941835:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781015466896 time 781030518849 state 0x0 ref 1 00000100:00100000:3.0:1789760801.941848:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781030779265 time 781030518849 state 0x1 ref 1 00000100:00100000:3.0:1789760801.941874:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781030779265 time 781046585096 state 0x0 ref 1 00000100:00100000:3.0:1789760801.941887:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508519936:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.941942:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26591232, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760801.941954:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57893 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.941968:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760801.941973:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.941979:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57893 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.941998:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57893 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.960893:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973812 is committed 00002000:00010000:3.0:1789760801.960929:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26591232 -> 26595328 00010000:00010000:3.0:1789760801.960945:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57893 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.960958:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57893 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.960972:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57893 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.960980:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.960986:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57893 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.960995:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57893 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.961014:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57893 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.961023:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57893 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26591232->26595327] (req 26591232->26595327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.961113:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508519936:12345-192.168.203.23@tcp:4:dd.0 Request processed in 19224us (19398us total) trans 4294973812 rc 0/0 00000100:00100000:3.0:1789760801.961124:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6524 00000100:00100000:3.0:1789760801.961134:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781030779265 time 781046585096 state 0x2 ref 0 00000100:00100000:0.0:1789760801.963948:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.964020:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508520064 00000100:00100000:3.0:1789760801.964031:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508520064 00000100:00100000:3.0:1789760801.964081:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781030779265 time 781046585096 state 0x0 ref 1 00000100:00100000:3.0:1789760801.964094:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781046845512 time 781046585096 state 0x1 ref 1 00000100:00100000:3.0:1789760801.964130:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781046845512 time 781068841140 state 0x0 ref 1 00000100:00100000:3.0:1789760801.964152:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508520064:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.964199:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26595328, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760801.964208:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5789a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.964222:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760801.964226:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.964231:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5789a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.964245:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5789a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760801.975996:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26595328 -> 26599424 00010000:00010000:3.0:1789760801.976024:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5789a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.976026:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973813 is committed 00010000:00010000:3.0:1789760801.976037:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5789a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.976043:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5789a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.976048:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.976052:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5789a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.976057:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5789a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.976076:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5789a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.976080:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5789a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26595328->26599423] (req 26595328->26599423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.976152:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508520064:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12002us (12210us total) trans 4294973813 rc 0/0 00000100:00100000:3.0:1789760801.976160:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6525 00000100:00100000:3.0:1789760801.976167:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781046845512 time 781068841140 state 0x2 ref 0 00000100:00100000:0.0:1789760801.978230:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.978458:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508520192 00000100:00100000:3.0:1789760801.978466:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508520192 00000100:00100000:2.0:1789760801.978566:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781046845512 time 781068841140 state 0x0 ref 1 00000100:00100000:2.0:1789760801.978580:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781069101556 time 781068841140 state 0x1 ref 1 00000100:00100000:2.0:1789760801.978609:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781069101556 time 781083320326 state 0x0 ref 1 00000100:00100000:2.0:1789760801.978690:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499c1ff450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508520192:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:2.0:1789760801.978732:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26599424, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:2.0:1789760801.978740:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe578a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.978754:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:2.0:1789760801.978758:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760801.978764:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe578a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760801.978779:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe578a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760801.988942:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973814 is committed 00002000:00010000:3.0:1789760801.989019:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26599424 -> 26603520 00010000:00010000:3.0:1789760801.989033:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe578a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.989046:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe578a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.989055:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe578a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.989062:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa0499c292600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760801.989067:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe578a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.989082:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe578a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.989102:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe578a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.989109:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000694fbf8e/0xb49177c0efe578a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26599424->26603519] (req 26599424->26603519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760801.989200:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499c1ff450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508520192:12345-192.168.203.23@tcp:4:dd.0 Request processed in 10511us (10969us total) trans 4294973814 rc 0/0 00000100:00100000:3.0:1789760801.989210:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6526 00000100:00100000:3.0:1789760801.989219:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781069101556 time 781083320326 state 0x2 ref 0 00000100:00100000:1.0:1789760801.991271:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760801.991310:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508520320 00000100:00100000:3.0:1789760801.991319:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508520320 00000100:00100000:3.0:1789760801.991365:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781069101556 time 781083320326 state 0x0 ref 1 00000100:00100000:3.0:1789760801.991379:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781083580742 time 781083320326 state 0x1 ref 1 00000100:00100000:3.0:1789760801.991415:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781083580742 time 781096119112 state 0x0 ref 1 00000100:00100000:3.0:1789760801.991429:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508520320:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760801.991467:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26603520, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760801.991474:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.991489:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760801.991493:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760801.991499:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760801.991511:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760802.000678:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26603520 -> 26607616 00000001:00080000:0.0:1789760802.000689:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973815 is committed 00010000:00010000:3.0:1789760802.000696:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.000710:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.000720:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.000728:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.000733:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.000741:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.000759:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.000768:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26603520->26607615] (req 26603520->26607615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.000849:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508520320:12345-192.168.203.23@tcp:4:dd.0 Request processed in 9418us (9578us total) trans 4294973815 rc 0/0 00000100:00100000:3.0:1789760802.000859:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6527 00000100:00100000:3.0:1789760802.000869:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781083580742 time 781096119112 state 0x2 ref 0 00000100:00100000:0.0:1789760802.002503:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.002541:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508520448 00000100:00100000:3.0:1789760802.002551:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508520448 00000100:00100000:3.0:1789760802.002596:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781083580742 time 781096119112 state 0x0 ref 1 00000100:00100000:3.0:1789760802.002611:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781096379528 time 781096119112 state 0x1 ref 1 00000100:00100000:3.0:1789760802.002647:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781096379528 time 781107356906 state 0x0 ref 1 00000100:00100000:3.0:1789760802.002659:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508520448:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.002697:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26607616, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.002704:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.002719:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.002723:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.002729:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.002745:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760802.012898:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973816 is committed 00002000:00010000:3.0:1789760802.012933:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26607616 -> 26611712 00010000:00010000:3.0:1789760802.012947:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.012960:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.012975:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.012982:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.012987:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.012996:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.013015:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.013023:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26607616->26611711] (req 26607616->26611711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.013110:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508520448:12345-192.168.203.23@tcp:4:dd.0 Request processed in 10444us (10602us total) trans 4294973816 rc 0/0 00000100:00100000:3.0:1789760802.013120:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6528 00000100:00100000:3.0:1789760802.013129:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781096379528 time 781107356906 state 0x2 ref 0 00000100:00100000:0.0:1789760802.014865:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.014897:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508520576 00000100:00100000:3.0:1789760802.014906:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508520576 00000100:00100000:3.0:1789760802.014950:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781096379528 time 781107356906 state 0x0 ref 1 00000100:00100000:3.0:1789760802.014964:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781107617322 time 781107356906 state 0x1 ref 1 00000100:00100000:3.0:1789760802.014989:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781107617322 time 781119699704 state 0x0 ref 1 00000100:00100000:3.0:1789760802.015002:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508520576:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.015042:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26611712, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.015051:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.015065:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.015069:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.015076:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.015090:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760802.024124:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973817 is committed 00002000:00010000:3.0:1789760802.024148:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26611712 -> 26615808 00010000:00010000:3.0:1789760802.024163:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.024176:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.024184:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.024191:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.024196:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.024202:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.024218:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.024222:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26611712->26615807] (req 26611712->26615807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.024332:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508520576:12345-192.168.203.23@tcp:4:dd.0 Request processed in 9328us (9464us total) trans 4294973817 rc 0/0 00000100:00100000:3.0:1789760802.024345:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6529 00000100:00100000:3.0:1789760802.024357:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781107617322 time 781119699704 state 0x2 ref 0 00000100:00100000:1.0:1789760802.026181:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.026250:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508520704 00000100:00100000:3.0:1789760802.026260:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508520704 00000100:00100000:3.0:1789760802.026327:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781107617322 time 781119699704 state 0x0 ref 1 00000100:00100000:3.0:1789760802.026341:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781119960120 time 781119699704 state 0x1 ref 1 00000100:00100000:3.0:1789760802.026363:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781119960120 time 781131073921 state 0x0 ref 1 00000100:00100000:3.0:1789760802.026386:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508520704:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.026425:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26615808, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.026433:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.026447:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.026451:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.026457:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.026470:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760802.037459:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973818 is committed 00002000:00010000:3.0:1789760802.037480:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26615808 -> 26619904 00010000:00010000:3.0:1789760802.037495:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.037505:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.037511:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.037515:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.037518:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.037524:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.037538:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.037542:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26615808->26619903] (req 26615808->26619903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.037608:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508520704:12345-192.168.203.23@tcp:4:dd.0 Request processed in 11232us (11430us total) trans 4294973818 rc 0/0 00000100:00100000:3.0:1789760802.037615:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6530 00000100:00100000:3.0:1789760802.037637:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781119960120 time 781131073921 state 0x2 ref 0 00000100:00100000:0.0:1789760802.039384:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.039473:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508520832 00000100:00100000:3.0:1789760802.039483:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508520832 00000100:00100000:3.0:1789760802.039527:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781119960120 time 781131073921 state 0x0 ref 1 00000100:00100000:3.0:1789760802.039541:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781131334337 time 781131073921 state 0x1 ref 1 00000100:00100000:3.0:1789760802.039573:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781131334337 time 781144283474 state 0x0 ref 1 00000100:00100000:3.0:1789760802.039585:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508520832:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.039636:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26619904, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.039644:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.039657:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.039661:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.039667:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.039679:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760802.051679:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973819 is committed 00002000:00010000:3.0:1789760802.051706:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26619904 -> 26624000 00010000:00010000:3.0:1789760802.051722:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.051736:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.051750:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.051757:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.051762:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.051771:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.051796:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.051805:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26619904->26623999] (req 26619904->26623999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.051887:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508520832:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12299us (12505us total) trans 4294973819 rc 0/0 00000100:00100000:3.0:1789760802.051898:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6531 00000100:00100000:3.0:1789760802.051906:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781131334337 time 781144283474 state 0x2 ref 0 00000100:00100000:0.0:1789760802.053584:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.053616:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508520960 00000100:00100000:3.0:1789760802.053633:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508520960 00000100:00100000:3.0:1789760802.053671:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781131334337 time 781144283474 state 0x0 ref 1 00000100:00100000:3.0:1789760802.053680:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781144543890 time 781144283474 state 0x1 ref 1 00000100:00100000:3.0:1789760802.053698:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781144543890 time 781158408582 state 0x0 ref 1 00000100:00100000:3.0:1789760802.053709:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508520960:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.053746:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26624000, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.053753:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.053766:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.053770:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.053776:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.053790:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760802.064639:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973820 is committed 00002000:00010000:3.0:1789760802.064717:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26624000 -> 26628096 00010000:00010000:3.0:1789760802.064732:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.064743:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.064751:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.064758:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.064762:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.064771:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.064789:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.064796:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26624000->26628095] (req 26624000->26628095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.064873:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508520960:12345-192.168.203.23@tcp:4:dd.0 Request processed in 11161us (11287us total) trans 4294973820 rc 0/0 00000100:00100000:3.0:1789760802.064882:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6532 00000100:00100000:3.0:1789760802.064891:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781144543890 time 781158408582 state 0x2 ref 0 00000100:00100000:1.0:1789760802.066469:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.066511:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508521088 00000100:00100000:3.0:1789760802.066520:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508521088 00000100:00100000:3.0:1789760802.066580:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781144543890 time 781158408582 state 0x0 ref 1 00000100:00100000:3.0:1789760802.066595:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781158668998 time 781158408582 state 0x1 ref 1 00000100:00100000:3.0:1789760802.066635:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781158668998 time 781171330662 state 0x0 ref 1 00000100:00100000:3.0:1789760802.066647:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508521088:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.066686:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26628096, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.066694:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.066707:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.066711:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.066717:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.066729:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760802.080723:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973821 is committed 00002000:00010000:3.0:1789760802.080801:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26628096 -> 26632192 00010000:00010000:3.0:1789760802.080814:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.080823:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.080829:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.080834:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.080837:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.080844:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.080861:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.080866:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26628096->26632191] (req 26628096->26632191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.080950:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508521088:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14301us (14482us total) trans 4294973821 rc 0/0 00000100:00100000:3.0:1789760802.080960:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6533 00000100:00100000:3.0:1789760802.080969:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781158668998 time 781171330662 state 0x2 ref 0 00000100:00100000:0.0:1789760802.082805:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.082858:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508521216 00000100:00100000:3.0:1789760802.082869:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508521216 00000100:00100000:3.0:1789760802.082915:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781158668998 time 781171330662 state 0x0 ref 1 00000100:00100000:3.0:1789760802.082929:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781171591078 time 781171330662 state 0x1 ref 1 00000100:00100000:3.0:1789760802.082954:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781171591078 time 781187664685 state 0x0 ref 1 00000100:00100000:3.0:1789760802.082967:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508521216:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.083016:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26632192, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.083024:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.083039:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.083044:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.083050:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.083063:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760802.094867:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973822 is committed 00002000:00010000:3.0:1789760802.094895:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26632192 -> 26636288 00010000:00010000:3.0:1789760802.094910:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.094924:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.094934:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.094941:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.094946:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.094956:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.094975:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.094992:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26632192->26636287] (req 26632192->26636287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.095073:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508521216:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12104us (12270us total) trans 4294973822 rc 0/0 00000100:00100000:3.0:1789760802.095084:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6534 00000100:00100000:3.0:1789760802.095094:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781171591078 time 781187664685 state 0x2 ref 0 00000100:00100000:0.0:1789760802.097371:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.097435:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508521344 00000100:00100000:3.0:1789760802.097451:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508521344 00000100:00100000:3.0:1789760802.097519:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781171591078 time 781187664685 state 0x0 ref 1 00000100:00100000:3.0:1789760802.097538:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781187925101 time 781187664685 state 0x1 ref 1 00000100:00100000:3.0:1789760802.097571:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781187925101 time 781202281473 state 0x0 ref 1 00000100:00100000:3.0:1789760802.097588:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508521344:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.097656:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26636288, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.097666:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.097694:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.097702:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.097711:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.097728:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760802.109532:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973823 is committed 00002000:00010000:3.0:1789760802.109574:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26636288 -> 26640384 00010000:00010000:3.0:1789760802.109594:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.109609:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.109638:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.109647:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.109653:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.109663:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.109689:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.109698:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26636288->26640383] (req 26636288->26640383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.109818:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508521344:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12228us (12446us total) trans 4294973823 rc 0/0 00000100:00100000:3.0:1789760802.109834:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6535 00000100:00100000:3.0:1789760802.109847:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781187925101 time 781202281473 state 0x2 ref 0 00000100:00100000:1.0:1789760802.111481:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.111541:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508521472 00000100:00100000:3.0:1789760802.111554:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508521472 00000100:00100000:3.0:1789760802.111611:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781187925101 time 781202281473 state 0x0 ref 1 00000100:00100000:3.0:1789760802.111641:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781202541889 time 781202281473 state 0x1 ref 1 00000100:00100000:3.0:1789760802.111670:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781202541889 time 781216380469 state 0x0 ref 1 00000100:00100000:3.0:1789760802.111686:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508521472:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.111739:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26640384, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.111747:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.111765:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.111770:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.111778:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.111793:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760802.123106:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973824 is committed 00002000:00010000:3.0:1789760802.123146:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26640384 -> 26644480 00010000:00010000:3.0:1789760802.123166:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.123178:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.123188:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.123196:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.123202:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.123212:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.123235:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.123241:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26640384->26644479] (req 26640384->26644479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.123382:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884219500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508521472:12345-192.168.203.23@tcp:4:dd.0 Request processed in 11691us (11900us total) trans 4294973824 rc 0/0 00000100:00100000:3.0:1789760802.123393:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6536 00000100:00100000:3.0:1789760802.123403:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781202541889 time 781216380469 state 0x2 ref 0 00000100:00100000:0.0:1789760802.124757:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.124819:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508521600 00000100:00100000:3.0:1789760802.124837:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508521600 00000100:00100000:3.0:1789760802.124899:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781202541889 time 781216380469 state 0x0 ref 1 00000100:00100000:3.0:1789760802.124917:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781216640885 time 781216380469 state 0x1 ref 1 00000100:00100000:3.0:1789760802.124945:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781216640885 time 781229655652 state 0x0 ref 1 00000100:00100000:3.0:1789760802.124962:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508521600:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.125015:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26644480, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.125023:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.125039:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.125044:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.125051:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.125067:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760802.134164:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973825 is committed 00002000:00010000:3.0:1789760802.134266:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26644480 -> 26648576 00010000:00010000:3.0:1789760802.134309:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.134324:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.134340:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.134348:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.134353:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.134362:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.134386:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.134394:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26644480->26648575] (req 26644480->26648575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.134501:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508521600:12345-192.168.203.23@tcp:4:dd.0 Request processed in 9538us (9743us total) trans 4294973825 rc 0/0 00000100:00100000:3.0:1789760802.134513:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6537 00000100:00100000:3.0:1789760802.134524:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781216640885 time 781229655652 state 0x2 ref 0 00000100:00100000:0.0:1789760802.136090:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.136146:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508521728 00000100:00100000:3.0:1789760802.136159:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508521728 00000100:00100000:3.0:1789760802.136210:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781216640885 time 781229655652 state 0x0 ref 1 00000100:00100000:3.0:1789760802.136224:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781229916068 time 781229655652 state 0x1 ref 1 00000100:00100000:3.0:1789760802.136251:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781229916068 time 781240961825 state 0x0 ref 1 00000100:00100000:3.0:1789760802.136263:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508521728:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.136339:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26648576, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.136348:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.136362:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.136367:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.136373:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.136387:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760802.147110:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973826 is committed 00002000:00010000:3.0:1789760802.147277:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26648576 -> 26652672 00010000:00010000:3.0:1789760802.147338:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.147353:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.147363:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.147372:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.147378:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.147389:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.147413:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.147422:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe578f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26648576->26652671] (req 26648576->26652671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.147540:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508521728:12345-192.168.203.23@tcp:4:dd.0 Request processed in 11272us (11448us total) trans 4294973826 rc 0/0 00000100:00100000:3.0:1789760802.147553:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6538 00000100:00100000:3.0:1789760802.147565:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781229916068 time 781240961825 state 0x2 ref 0 00000100:00100000:1.0:1789760802.149262:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.149332:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508521856 00000100:00100000:3.0:1789760802.149345:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508521856 00000100:00100000:3.0:1789760802.149407:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781229916068 time 781240961825 state 0x0 ref 1 00000100:00100000:3.0:1789760802.149425:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781241222241 time 781240961825 state 0x1 ref 1 00000100:00100000:3.0:1789760802.149454:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781241222241 time 781254164777 state 0x0 ref 1 00000100:00100000:3.0:1789760802.149469:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508521856:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.149519:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26652672, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.149529:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.149558:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.149564:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.149572:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.149588:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760802.163529:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973827 is committed 00002000:00010000:3.0:1789760802.163563:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26652672 -> 26656768 00010000:00010000:3.0:1789760802.163578:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.163591:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.163600:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.163607:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.163611:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.163633:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.163658:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.163667:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe578fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26652672->26656767] (req 26652672->26656767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.163758:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508521856:12345-192.168.203.23@tcp:4:dd.0 Request processed in 14288us (14498us total) trans 4294973827 rc 0/0 00000100:00100000:3.0:1789760802.163767:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6539 00000100:00100000:3.0:1789760802.163778:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781241222241 time 781254164777 state 0x2 ref 0 00000100:00100000:0.0:1789760802.165630:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.165749:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508521984 00000100:00100000:3.0:1789760802.165760:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508521984 00000100:00100000:3.0:1789760802.165807:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781241222241 time 781254164777 state 0x0 ref 1 00000100:00100000:3.0:1789760802.165822:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781254425193 time 781254164777 state 0x1 ref 1 00000100:00100000:3.0:1789760802.165846:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781254425193 time 781270556774 state 0x0 ref 1 00000100:00100000:3.0:1789760802.165857:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508521984:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.165895:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26656768, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.165903:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57903 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.165918:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.165924:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.165930:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57903 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.165942:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57903 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760802.176007:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973828 is committed 00002000:00010000:3.0:1789760802.176023:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26656768 -> 26660864 00010000:00010000:3.0:1789760802.176039:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57903 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.176052:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57903 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.176061:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57903 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.176068:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.176072:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57903 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.176081:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57903 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.176099:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57903 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.176106:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57903 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26656768->26660863] (req 26656768->26660863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.176184:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508521984:12345-192.168.203.23@tcp:4:dd.0 Request processed in 10325us (10566us total) trans 4294973828 rc 0/0 00000100:00100000:3.0:1789760802.176194:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6540 00000100:00100000:3.0:1789760802.176203:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781254425193 time 781270556774 state 0x2 ref 0 00000100:00100000:0.0:1789760802.177705:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.177733:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508522112 00000100:00100000:3.0:1789760802.177743:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508522112 00000100:00100000:3.0:1789760802.177783:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781254425193 time 781270556774 state 0x0 ref 1 00000100:00100000:3.0:1789760802.177796:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781270817190 time 781270556774 state 0x1 ref 1 00000100:00100000:3.0:1789760802.177821:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781270817190 time 781282531619 state 0x0 ref 1 00000100:00100000:3.0:1789760802.177834:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887fc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508522112:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.177872:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26660864, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.177879:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5790a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.177900:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.177905:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.177911:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5790a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.177924:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5790a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760802.188967:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973829 is committed 00002000:00010000:3.0:1789760802.188973:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26660864 -> 26664960 00010000:00010000:3.0:1789760802.188989:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5790a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.189003:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5790a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.189012:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5790a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.189017:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.189021:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5790a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.189027:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5790a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.189041:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5790a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.189046:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe5790a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26660864->26664959] (req 26660864->26664959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.189122:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887fc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508522112:12345-192.168.203.23@tcp:4:dd.0 Request processed in 11286us (11418us total) trans 4294973829 rc 0/0 00000100:00100000:3.0:1789760802.189135:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6541 00000100:00100000:3.0:1789760802.189144:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781270817190 time 781282531619 state 0x2 ref 0 00000100:00100000:1.0:1789760802.190737:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.190771:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508522240 00000100:00100000:3.0:1789760802.190779:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508522240 00000100:00100000:3.0:1789760802.190815:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781270817190 time 781282531619 state 0x0 ref 1 00000100:00100000:3.0:1789760802.190825:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781282792035 time 781282531619 state 0x1 ref 1 00000100:00100000:3.0:1789760802.190844:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781282792035 time 781295555639 state 0x0 ref 1 00000100:00100000:3.0:1789760802.190851:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508522240:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.190884:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26664960, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.190891:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57911 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.190903:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.190907:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.190913:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57911 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.190925:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57911 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760802.199968:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973830 is committed 00002000:00010000:3.0:1789760802.199999:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26664960 -> 26669056 00010000:00010000:3.0:1789760802.200015:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57911 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.200029:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57911 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.200038:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57911 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.200046:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:3.0:1789760802.200050:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57911 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.200061:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57911 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.200082:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57911 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.200091:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe57911 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26664960->26669055] (req 26664960->26669055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760802.200178:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04884218380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2406:x1876699508522240:12345-192.168.203.23@tcp:4:dd.0 Request processed in 9322us (9440us total) trans 4294973830 rc 0/0 00000100:00100000:3.0:1789760802.200189:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6542 00000100:00100000:3.0:1789760802.200199:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781282792035 time 781295555639 state 0x2 ref 0 00000100:00100000:0.0:1789760802.201859:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.201898:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508522368 00000100:00100000:3.0:1789760802.201907:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508522368 00000100:00100000:3.0:1789760802.201954:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781282792035 time 781295555639 state 0x0 ref 1 00000100:00100000:3.0:1789760802.201967:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781295816055 time 781295555639 state 0x1 ref 1 00000100:00100000:3.0:1789760802.201987:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781295816055 time 781306697952 state 0x0 ref 1 00000100:00100000:3.0:1789760802.201999:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508522368:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.202036:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26669056, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.202043:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57918 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.202056:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.202060:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.202067:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57918 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.202080:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57918 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760802.210797:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973831 is committed 00002000:00010000:0.0:1789760802.210829:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26669056 -> 26673152 00010000:00010000:0.0:1789760802.210845:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57918 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.210860:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57918 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.210869:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57918 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.210876:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb1600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760802.210880:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57918 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.210890:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57918 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.210909:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57918 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.210914:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b87bea4/0xb49177c0efe57918 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26669056->26673151] (req 26669056->26673151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760802.211002:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2407:x1876699508522368:12345-192.168.203.23@tcp:4:dd.0 Request processed in 9000us (9146us total) trans 4294973831 rc 0/0 00000100:00100000:0.0:1789760802.211012:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6543 00000100:00100000:0.0:1789760802.211021:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781295816055 time 781306697952 state 0x2 ref 0 00000100:00100000:0.0:1789760802.212862:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.212916:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508522496 00000100:00100000:3.0:1789760802.212925:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508522496 00000100:00100000:3.0:1789760802.212973:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781295816055 time 781306697952 state 0x0 ref 1 00000100:00100000:3.0:1789760802.212988:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781306958368 time 781306697952 state 0x1 ref 1 00000100:00100000:3.0:1789760802.213012:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781306958368 time 781317722773 state 0x0 ref 1 00000100:00100000:3.0:1789760802.213029:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508522496:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.213072:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26673152, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.213080:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5791f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.213093:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.213097:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.213103:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5791f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.213116:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5791f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760802.223039:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973832 is committed 00002000:00010000:0.0:1789760802.223108:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26673152 -> 26677248 00010000:00010000:0.0:1789760802.223123:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5791f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.223132:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5791f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.223138:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5791f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.223142:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0a00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760802.223145:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5791f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.223152:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5791f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.223167:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5791f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.223171:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000afcbdc22/0xb49177c0efe5791f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26673152->26677247] (req 26673152->26677247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760802.223245:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499ca5dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508522496:12345-192.168.203.23@tcp:4:dd.0 Request processed in 10214us (10383us total) trans 4294973832 rc 0/0 00000100:00100000:0.0:1789760802.223255:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6544 00000100:00100000:0.0:1789760802.223264:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781306958368 time 781317722773 state 0x2 ref 0 00000100:00100000:1.0:1789760802.225376:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:3.0:1789760802.225469:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508522624 00000100:00100000:3.0:1789760802.225479:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508522624 00000100:00100000:3.0:1789760802.225526:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781306958368 time 781317722773 state 0x0 ref 1 00000100:00100000:3.0:1789760802.225541:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781317983189 time 781317722773 state 0x1 ref 1 00000100:00100000:3.0:1789760802.225569:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781317983189 time 781330279588 state 0x0 ref 1 00000100:00100000:3.0:1789760802.225584:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508522624:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:3.0:1789760802.225636:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26677248, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:3.0:1789760802.225645:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57926 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.225667:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:3.0:1789760802.225671:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760802.225677:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57926 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760802.225689:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57926 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760802.236109:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973833 is committed 00002000:00010000:0.0:1789760802.236324:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26677248 -> 26681344 00010000:00010000:0.0:1789760802.236337:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57926 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.236349:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57926 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.236357:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57926 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.236365:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa049b7eb0c00 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760802.236370:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57926 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.236379:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57926 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.236399:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57926 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.236411:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004cda59ee/0xb49177c0efe57926 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26677248->26681343] (req 26677248->26681343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760802.236492:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0488421b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508522624:12345-192.168.203.23@tcp:4:dd.0 Request processed in 10908us (11118us total) trans 4294973833 rc 0/0 00000100:00100000:0.0:1789760802.236500:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6545 00000100:00100000:0.0:1789760802.236507:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781317983189 time 781330279588 state 0x2 ref 0 00000100:00100000:3.0:1789760802.238720:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760802.238772:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508522752 00000100:00100000:0.0:1789760802.238782:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508522752 00000100:00100000:0.0:1789760802.238828:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781317983189 time 781330279588 state 0x0 ref 1 00000100:00100000:0.0:1789760802.238842:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781330540004 time 781330279588 state 0x1 ref 1 00000100:00100000:0.0:1789760802.238867:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781330540004 time 781343577485 state 0x0 ref 1 00000100:00100000:0.0:1789760802.238880:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508522752:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760802.238922:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26681344, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:0.0:1789760802.238930:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5792d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.238945:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760802.238949:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760802.238955:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5792d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.238975:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5792d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760802.251494:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973834 is committed 00002000:00010000:0.0:1789760802.251533:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26681344 -> 26685440 00010000:00010000:0.0:1789760802.251548:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5792d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.251567:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5792d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.251575:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5792d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.251583:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760802.251588:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5792d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.251597:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5792d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.251615:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5792d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.251631:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5792d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26681344->26685439] (req 26681344->26685439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760802.251713:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508522752:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12832us (12995us total) trans 4294973834 rc 0/0 00000100:00100000:0.0:1789760802.251721:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6546 00000100:00100000:0.0:1789760802.251728:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781330540004 time 781343577485 state 0x2 ref 0 00000100:00100000:3.0:1789760802.253606:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760802.253655:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508522880 00000100:00100000:0.0:1789760802.253675:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508522880 00000100:00100000:0.0:1789760802.253717:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781330540004 time 781343577485 state 0x0 ref 1 00000100:00100000:0.0:1789760802.253731:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781343837901 time 781343577485 state 0x1 ref 1 00000100:00100000:0.0:1789760802.253755:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781343837901 time 781358466225 state 0x0 ref 1 00000100:00100000:0.0:1789760802.253766:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508522880:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760802.253803:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26685440, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:0.0:1789760802.253811:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57934 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.253824:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760802.253828:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760802.253834:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57934 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.253852:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57934 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760802.263315:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973835 is committed 00002000:00010000:0.0:1789760802.263340:0:8109:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26685440 -> 26689536 00010000:00010000:0.0:1789760802.263352:0:8109:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57934 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.263361:0:8109:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57934 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.263371:0:8109:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57934 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.263375:0:8109:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760802.263379:0:8109:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57934 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.263384:0:8109:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57934 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.263398:0:8109:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57934 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.263402:0:8109:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57934 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26685440->26689535] (req 26685440->26689535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760802.263473:0:8109:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508522880:12345-192.168.203.23@tcp:4:dd.0 Request processed in 9704us (9865us total) trans 4294973835 rc 0/0 00000100:00100000:0.0:1789760802.263483:0:8109:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6547 00000100:00100000:0.0:1789760802.263491:0:8109:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781343837901 time 781358466225 state 0x2 ref 0 00000100:00100000:1.0:1789760802.265492:0:3465:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760802.265540:0:14639:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508523008 00000100:00100000:0.0:1789760802.265549:0:14639:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508523008 00000100:00100000:0.0:1789760802.265591:0:14639:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781343837901 time 781358466225 state 0x0 ref 1 00000100:00100000:0.0:1789760802.265604:0:14639:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781358726641 time 781358466225 state 0x1 ref 1 00000100:00100000:0.0:1789760802.265644:0:14639:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781358726641 time 781370354445 state 0x0 ref 1 00000100:00100000:0.0:1789760802.265660:0:14639:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508523008:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760802.265699:0:14639:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26689536, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:0.0:1789760802.265707:0:14639:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5793b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.265720:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760802.265724:0:14639:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760802.265731:0:14639:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5793b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.265744:0:14639:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5793b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760802.277493:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973836 is committed 00002000:00010000:0.0:1789760802.277582:0:14639:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26689536 -> 26693632 00010000:00010000:0.0:1789760802.277595:0:14639:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5793b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.277604:0:14639:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5793b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.277610:0:14639:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5793b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.277615:0:14639:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031200 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760802.277628:0:14639:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5793b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.277635:0:14639:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5793b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.277650:0:14639:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5793b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.277655:0:14639:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe5793b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26689536->26693631] (req 26689536->26693631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14639 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760802.277729:0:14639:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa04887f30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508523008:12345-192.168.203.23@tcp:4:dd.0 Request processed in 12070us (12237us total) trans 4294973836 rc 0/0 00000100:00100000:0.0:1789760802.277739:0:14639:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6548 00000100:00100000:0.0:1789760802.277748:0:14639:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781358726641 time 781370354445 state 0x2 ref 0 00000100:00100000:3.0:1789760802.279534:0:3467:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760802.279585:0:8108:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508523136 00000100:00100000:0.0:1789760802.279596:0:8108:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508523136 00000100:00100000:0.0:1789760802.279658:0:8108:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781358726641 time 781370354445 state 0x0 ref 1 00000100:00100000:0.0:1789760802.279671:0:8108:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781370614861 time 781370354445 state 0x1 ref 1 00000100:00100000:0.0:1789760802.279697:0:8108:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781370614861 time 781384408174 state 0x0 ref 1 00000100:00100000:0.0:1789760802.279708:0:8108:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508523136:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760802.279744:0:8108:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26693632, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:0.0:1789760802.279753:0:8108:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57942 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.279766:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760802.279771:0:8108:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760802.279777:0:8108:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57942 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.279786:0:8108:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57942 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760802.290575:0:8134:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294973837 is committed 00002000:00010000:0.0:1789760802.290632:0:8108:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x3:0x0] updating lvb size from disk: 26693632 -> 26697728 00010000:00010000:0.0:1789760802.290653:0:8108:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57942 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.290668:0:8108:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57942 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.290678:0:8108:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57942 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.290686:0:8108:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffffa04980031600 already unused, calling callback (ffffffffc1532350) 00010000:00010000:0.0:1789760802.290691:0:8108:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57942 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.290702:0:8108:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57942 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.290731:0:8108:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57942 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.290740:0:8108:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077d51617/0xb49177c0efe57942 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26693632->26697727] (req 26693632->26697727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8108 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760802.290848:0:8108:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffffa0499b6c3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2409:x1876699508523136:12345-192.168.203.23@tcp:4:dd.0 Request processed in 11135us (11317us total) trans 4294973837 rc 0/0 00000100:00100000:0.0:1789760802.290860:0:8108:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.23@tcp, seq: 6549 00000100:00100000:0.0:1789760802.290872:0:8108:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781370614861 time 781384408174 state 0x2 ref 0 00000100:00100000:3.0:1789760802.292386:0:3466:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.23@tcp (source: 12345-192.168.203.23@tcp) 00000100:00100000:0.0:1789760802.292429:0:8109:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699508523264 00000100:00100000:0.0:1789760802.292439:0:8109:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699508523264 00000100:00100000:0.0:1789760802.292509:0:8109:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781370614861 time 781384408174 state 0x0 ref 1 00000100:00100000:0.0:1789760802.292526:0:8109:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781384668590 time 781384408174 state 0x1 ref 1 00000100:00100000:0.0:1789760802.292553:0:8109:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffffa04884428400 key nid={192.168.203.23@tcp}&opcode={ost_write} rule@0xffffa049be4f6800 gen 0 rate 15 depth 3 token 3 deadline 781384668590 time 781397263756 state 0x0 ref 1 00000100:00100000:0.0:1789760802.292567:0:8109:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffffa0499b6c2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:4f8f2316-7bfa-4328-bbb6-ea882f587c18+5:2408:x1876699508523264:12345-192.168.203.23@tcp:4:dd.0 00002000:00010000:0.0:1789760802.292608:0:8109:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x3:0x0] initial LVB size: 26697728, mtime: 0x6aad9522, atime: 0x0, ctime: 0x6aad9522, blocks: 0xd009 00010000:00010000:0.0:1789760802.292615:0:8109:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57949 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.292647:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=781s v=(0 0) 00010000:00010000:0.0:1789760802.292654:0:8109:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760802.292661:0:8109:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57949 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760802.292676:0:8109:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d32579f/0xb49177c0efe57949 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x3:0x0].0x0 rrc: 2 type: EXT [26697728->26701823] (req 26697728->26701823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8109 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 Debug log: 57177 lines, 57177 kept, 0 dropped, 0 bad.